DETAILED ACTION
Claim 6 cancelled by preliminary amendment.
Claims 1-5 and 7 rejected under 35 USC § 103.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Tariq et al., U.S. PG-Publication No. 2023/0071281 A1 (hereinafter TARIQ), in view of Fu et al., U.S. PG-Publication No. 2018/0101416 A1 (hereinafter FU), further in view of Van Bokhoven et al., U.S. PG-Publication No. 2011/0191609 A1 (hereinafter BOKHOVEN), further in view of Mahadik, U.S. PG-Publication No. 2023/0032853 A1 (hereinafter MAHADIK).
Claim 1
TARIQ discloses a scaling management device configured to manage scaling of
virtual resources mounted on hardware through a virtualization platform. ¶ 0011: Kubernetes and Kubeflow deploy processing tasks using multiple containers as parameter server and worker nodes. ¶ 0014: The controller automatically and adaptively allocates resources to processing tasks implemented in “a container format” or using VMs and makes scaling decisions based on monitored utilization. ¶ 0018: Each compute node may be “a bare metal machine, a virtual machine, or a container.”
TARIQ discloses the scaling management device comprising one or more processors configured to perform operations. ¶ 0021: The controller system and resource-allocation platform each include “one or more processors,” memory, and computer-readable media. ¶ 0025: The controller implements a scaling engine, stability detector, placement engine, resource-assignment optimization engine, and metric collector.
TARIQ discloses calculating the number of virtual resources to be mounted on the hardware. ¶ 0040: The controller generates configurations comprising a number of parameter server nodes and a number of worker nodes. ¶ 0041: A second configuration changes the containers from the first number of parameter server and worker nodes to second numbers.
TARIQ discloses calculating requested resources for the virtual resources to be mounted on the hardware. ¶ 0027: The distributed resource configuration contains container numbers nps,nwk and compute resource allocations rps,rwk. ¶ 0042: The compute resources allocated to an individual container are calculated by dividing a resource budget by the number of parameter server and worker nodes.
TARIQ discloses creating a scaling config file. ¶ 0028: The scaling engine adjusts a resource through a configurable scale-up ratio by updating the distributed resource configuration. ¶ 0050: The control plane stores one or more distributed resource configurations in a configuration data store.
TARIQ discloses creating a resource config file including the calculated number of virtual resources to be mounted on the hardware and the requested resources of the virtual resources to be mounted on the hardware. ¶ 0027: One configuration includes both nps,nwk representing container numbers and rps,rwk representing allocated compute resources. ¶ 0043: The resource allocation platform allocates the resource containers according to the generated configurations.
TARIQ discloses using an efficiency value to determine scaling and resource allocation. ¶¶ 0028-0029: Calculates overall compute resource efficiency and adjusts the configuration using a scaling ratio. ¶¶ 0065-0068: Low efficiency causes addition of worker or parameter server containers and redistribution of resources among the resulting nodes. However, TARIQ does not expressly disclose calculating electric power efficiency characteristics of each of a plurality of pieces of the hardware by measuring electric power efficiency of the hardware; measuring performance of each piece of hardware by measuring a predetermined metric; identifying the hardware having a lowest performance value; and calculating a performance value of other hardware as a performance ratio in a case where a performance value of the identified hardware is set to 1.
FU discloses calculating electric power efficiency characteristics of each of a plurality of pieces of the hardware by measuring electric power efficiency of the hardware. ¶ 0059: Energy efficiency is throughput divided by energy-consumption rate and represents useful work, including throughput or processes per second, per watt. ¶ 0054: A respective effective energy efficiency value is determined for each server from its service rate and the difference between busy and idle energy consumption rates. ¶ 0125: Throughput is measured in jobs per second, power consumption is measured in wars, and energy efficiency is jobs per watt-second.
FU discloses measuring performance of each piece of hardware by measuring a predetermined metric. ¶ 0054: Each server has a measurable service rate used in calculating its effective energy efficiency value. ¶ 0133: The settings of servers are “based on the benchmark results.”
FU discloses identifying the hardware having a lowest performance value, and calculating a performance value of other hardware as a performance ratio in a case where a performance value of the identified hardware is set to 1. ¶ 0133: The third server group has the baseline service rate, while the first and second groups have higher relative service rates. The third group’s (µ3) service rate “is normalized to one,” and the other service rate ratios are 3.5 (µ1/µ3) and 1.4 (µ2/µ3) relative to that baseline.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the container and VM scaling controller of TARIQ to incorporate the per-hardware work per-watt efficiency calculation and normalized hardware performance measurements taught by FU. One of ordinary skill in the art would be motivated to integrate FU’s efficiency and performance measurements into TARIQ with a reasonable expectation of success, in order to conserve energy and maximize effective energy efficiency when managing heterogenous servers (FU, ¶ 0004).
FU discloses using hardware specific efficiency values to control allocation. ¶ 0067: Selects the server having the highest effective energy efficiency among the available servers. ¶¶ 0072-0073: Defines effective efficiency using service rate divided by incremental busy-over-idle power and ranks hardware according to that value. However, TARIQ-FU does not expressly disclose calculating electric power efficiency characteristics of each of a plurality of pieces of the hardware by measuring electric power efficiency of the hardware while varying a level of a usage rate of the hardware; and determining a usage rate at which a value of electric power efficiency is highest among the measured electric power efficiencies.
BOKHOVEN discloses measuring electric power efficiency of the hardware while varying a level of a usage rate of the hardware. ¶ 0051: Actual power consumption is measured at several levels of total CPU usage by increasing usage in ten-percentage-point steps and measuring the corresponding power consumption. ¶ 0050: The relationship between total CPU usage and total power consumption is “clearly nonlinear.” ¶ 0052: The utilization/power characteristics should be determined for as many different hardware configurations and types as possible.
BOKHOVEN discloses determining a usage rate at which a value of electric power efficiency is highest among the measured electric power efficiencies. ¶ 0047: Utilization is associated with stored power-usage characteristics represented as tables or functions. ¶ 0051: Power measurements at ten-percent utilization increments produce the power character table and chart. ¶ 0052: The utilization/power characteristics should be determined for as many different hardware configurations and types as possible.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the power efficiency aware scaling controller of TARIQ-FU to incorporate the stepped, hardware specific utilization and power characterization procedure taught by BOKHOVEN. One of ordinary skill in the art would be motivated to integrate BOKHOVEN’s utilization and power characterization procedure into TARIQ-FU, with a reasonable expectation of success, in order to improve the accuracy of power consumption estimates across different hardware configurations and types, while enabling utilization measurements to be matched to power consumption easily and quickly (BOKHOVEN, ¶¶ 0047, 0052).
TARIQ-FU-BOKHAVEN does not expressly disclose calculating the number of virtual resources to be mounted on the hardware in accordance with a ratio of magnitude of the score calculated for each piece of hardware; calculating requested resources for the virtual resource to be mounted on the hardware having a lower performance value and the other hardware by setting the requested resource of the hardware having a lower performance value to a value indicated by the performance ratio to 1 in a case where the requested resource for the virtual resource to be mounted on the other hardware is set to 1 on the basis of the performance ratio of the other hardware to the performance value 1 of the hardware having a lowest performance value; and creating a scaling config file for each piece of hardware with the usage rate at which the value of electric power efficiency is highest as a target value of scaling of the hardware.
MAHADIK discloses calculating the number of virtual resources to be mounted on the hardware in accordance with a ratio of magnitude of the score calculated for each piece of hardware. ¶ 0018: A scalar reward metric corresponds one-to-one with each candidate container configuration, and reward magnitude is used to select the configuration. ¶ 0029: Each configuration may indicate the cardinality of the container set (cardinality → the number count of containers in the set, i.e. number of virtual resources to be mounted). ¶ 0020: Allocated CPU quantity may scale linearly with container cardinality (scale linearly → in accordance with a ratio of magnitude).
MAHADIK discloses calculating requested resources for the virtual resource to be mounted on the hardware having a lower performance value and the other hardware by setting the requested resource of the hardware having a lower performance value to a value indicated by the performance ratio to 1 in a case where the requested resource for the virtual resource to be mounted on the other hardware is set to 1 on the basis of the performance ratio of the other hardware to the performance value 1 of the hardware having a lowest performance value. ¶ 0020: A container configuration may specify the number of CPU devices allocated to each container, and the allocation may vary across the containers. ¶ 0029: Each candidate configuration identifies both the container cardinality and the corresponding allocation of computational resources. ¶ 0042: Configurations with less resource waste receive higher rewards and are preferred over configurations with greater resource waste.
MAHADIK discloses creating a scaling config file for each piece of hardware with the usage rate at which the value of electric power efficiency is highest as a target value of scaling of the hardware. ¶ 0021: Each container configuration includes utilization threshold that trigger scale-up and scale-down events. Scaling may change container cardinality or allocated resources. ¶ 0037: Each configuration may be encoded using CPU allocation, a lower utilization threshold, and an upper utilization threshold. ¶ 0043: The reward increases for configurations having more efficient utilization and decreases for configurations having lower utilization.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the hardware profiled, performance normalized, power efficient scaling controller of TARIQ-FU-BOKHOVEN to incorporate the efficiency metric based container configurations taught by MAHADIK, including configurations specifying container cardinality, per-container resource allocation, and utilization-based scaling thresholds. One of ordinary skill in the art would be motivated to integrate MAHADIK’s container configuration technique into TARIQ-FU-BOKHOVEN, with a reasonable expectation of success, in order to match computational resources to workload demand and thereby avoid both diminished processing speed from insufficient resources and resource wastage from excessive resources (MAHADIK, ¶ 0002).
Claim 2
TARIQ discloses acquiring a usage rate of the virtual resources mounted on the hardware in units of the hardware. ¶ 0015: During runtime, the system monitors “compute resource utilization of each compute node” and dynamically adjusts each compute node’s resource assignment. ¶ 0025: The controller includes a scaling engine, placement engine, resource-assignment optimization engine, and metric collector. ¶ 0047: the metric collector collects telemetry including compute and network utilization. ¶ 0055: While the task operates under its current resource allocation, the controller receives metric data including compute utilization. ¶ 0037: The placement engine determines the compute nodes launches as containers on particular cluster nodes. ¶ 0015: The system monitors utilization of each compute node and maps parameter server and worker nodes to physical servers (i.e., in units of the hardware).
TARIQ discloses determining addition … of the virtual resources. ¶ 0045: When resource efficiency is below the threshold, the controller generates a configuration that adds parameter-server or worker nodes. ¶ 0065: The controller responds to insufficient resource efficiency by adding worker nodes or parameter server nodes.
TARIQ does not expressly disclose determining deletion of the virtual resources by comparing the acquired usage rate of the virtual resources with a target value of
scaling indicated by the scaling config file.
MAHADIK discloses determining deletion of the virtual resources. ¶ 0021: A scaling event may horizontally scale the set of containers by increasing or decreasing the number of containers. ¶ 0040: When current utilization exceeds the upper utilization threshold, the containers are scaled up. When current utilization falls below the lower utilization threshold, the containers are scaled down. Horizontal scaling increases of decreases the number of containers.
MAHADIK discloses by comparing the acquired usage rate of the virtual resources with a target value of scaling indicated by the scaling config file. ¶ 0021: Each configuration indicates an upper utilization threshold and a lower utilization threshold. Current utilization is compared with those threshold to determine whether the containers should be scaled. ¶ 0037: A configuration includes a CPU allocation, a lower utilization threshold , and an upper utilization threshold. ¶ 0038: The utilization data represents the fraction of the allocated CPU resources used by the set of containers. ¶ 0040: The result of the comparison determines whether the number of containers is increased or decreased.
Claim 3
FU discloses determining hardware with low maximum power consumption or hardware with low standby power as hardware to be added. ¶ 0005: The method identifies servers able to accept additional work, calculates an effective energy efficiency value for each server, and assigns the work to the server having the highest value. The calculation uses service rate, power while busy, and power while idle. ¶ 0015: One or more processors determine available servers, determine their effective energy efficiency values, and assign work to the server having the highest value. ¶ 0054: The server management method identifies servers having available capacity, determines their effective energy efficiency values, sorts the servers, and selects the server having the highest value. ¶ 0056: The method may be combined with a right sizing technique in which idle servers are powered off. ¶ 0067: FU selects the available sever having the highest effective energy efficiency while accounting for idle power. ¶ 0072: FU distinguishes productive power used while a server is busy from unproductive power consumed while the server is idle.
TARIQ discloses adding hardware in a case where the addition of the virtual resources falls or a case where the acquired usage rate of the virtual resources is higher than a predetermined first threshold. ¶ 0061: After an increase in resources is requested, the controller determines that the resources available in the cluster are less than the requested amount. ¶ 0064: The controller determines whether increasing the resources has encountered a performance bottleneck because a compute node cannot efficiently use additional resources. ¶ 0066: The controller polls utilization samples for each parameter server node. When more than ten percent of the samples exceed ninety percent of network capacity, the controller determines that a bottleneck exists and adds a new parameter server node.
Claim 4
FU discloses determining the hardware with high maximum electric power or the hardware with high standby power as hardware to be stopped. ¶ 0015: One or more processors determine available servers, determine their effective energy efficiency values, and assign work according to those values. ¶ 0054: FU determines and sorts the effective energy efficiency values of available servers using service rate, power while busy, and power while idle. ¶ 0056: FU may be combined with a right sizing technique that powers off idle servers. ¶ 0067: FU selects the available server having the highest effective energy efficiency while accounting for idle power. ¶ 0072: Power consumed while a server is idle is unproductive and constitutes wasted energy.
TARIQ discloses stopping hardware in a case where the acquired usage rate of the virtual resources is lower than a predetermined second threshold. ¶ 0028: The scaling engine aggregates compute resource usage values and compares the resulting resource efficiency with a threshold when determining how the resource configuration should be changed. ¶ 0029: Resource efficiency is calculated as the sum of compute node usage divided by the sum of assigned resources and is compared with a threshold such as 0.7. ¶ 0015: The controller monitors compute resource utilization of each compute node during operation and makes adjustments to the resource assignments.
Claim 5
Claim 5 is rejected utilizing the rationale for claim 1; the claim is directed to a method performed by the system.
Claim 7
Claim 7 is rejected utilizing the rationale for claim 1; the claim is directed to a medium storing instructions executed by the system.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See Ansari et al., U.S. PG-Publication No. 2016/0216991 A1. ANSARI sorts physical machines according to power efficiency, selects a relatively more power efficient physical machine, and allocates the selected virtual machine to it (ANSARI, ¶¶ 0051-0054).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FRANK D MILLS whose telephone number is (571)270-3194. The examiner can normally be reached M-F 9-5:30 CT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, KEVIN YOUNG can be reached at (571)270-3180. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/FRANK D MILLS/Primary Examiner, Art Unit 2194 August 22, 2026
Prosecution Insights