Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-11 were canceled in a preliminary amendment filed on 6/11/2025.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 13-14, 21-22, and 29- 30 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 13, it recites “substituting the consumed video memory capacity of the plurality of GPUs, the total video memory capacity of each GPU of the plurality of GPUs, and the types and the quantity of the service instances running on each GPU of the plurality of GPUs into a plurality of predetermined inequalities corresponding to the plurality of GPUs.” It is unclear how a type (not a numerical value) is substituted into an inequality. The specification does not provide guidance on how a type can be substituted into an inequality (Specification para [0038] – [0039]). It is unclear what is meant by substituting the types into a plurality of predetermined inequalities and the claim takes on an unreasonable degree of uncertainty and is thus indefinite. For the purpose of compact prosecution and art rejection, the examiner will interpret the limitation in question to mean substituting variables that represent predicted video memory capacities consumed by different types of the service instances.
Regarding claim 21 and 29, they are rejected using the same citations and rationales described in the rejection of claim 13.
Regarding claims 14, 22, and 30, they are rejected due to being dependent on claims 13, 21, and 29 respectively.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 12, 15-20, 23-28 and 31 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sun et al. (CN114896070A; hereinafter Sun) in view of Fan (CN113064695A; hereinafter Fan).
Regarding claim 28, Sun teaches A computer-implemented system for scheduling services running on graphics processing units (GPUs) (“…a resource scheduling system supporting GPU sharing is provided…,” (page 23, para [n0043]).
“…a GPU resource allocation method for deep learning tasks is provided, applied to a resource scheduling system of a multi-tenant container cloud platform, the resource scheduling system comprising a control node and multiple worker nodes…,” (page 6, para [n0006]). ), comprising:
one or more computers (“The resource scheduling system includes a control node and multiple worker nodes, each worker node including multiple GPUs,” (page 16, para [n0028]). One or more computers includes multiple nodes and / or multiple processors.); and
one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations (“This invention can be a system, method, and/or computer program product. Computer program products may include computer-readable storage media having computer-readable program control instructions loaded thereon for causing a processor to implement various aspects of the present invention,” (pages 78-79, para [n0123]).
“A computer-readable storage medium can be a tangible device that holds and stores control instructions used by a control instruction execution device,” (pages 79-80, para [n0124]).), comprising:
obtaining, for each GPU of a plurality of GPUs (“According to a second aspect of the present invention, a GPU resource allocation method for deep learning tasks is provided, applied to a resource scheduling system of a multi-tenant container cloud platform. The resource scheduling system includes a control node and multiple worker nodes, each worker node including multiple GPUs,” (page 16, para [n0028]).
“…considering that there are multiple working nodes in the resource scheduling system, and each working node has multiple GPUs, the GPUs will have different computing power due to their different specifications. Therefore, in order to facilitate the allocation of appropriate GPU resources for deep learning tasks, when obtaining the available resource set, in addition to counting the remaining resources of each GPU in the working node, it is also necessary to count the computing power corresponding to each GPU. That is, in this embodiment of the application, the available resource set includes the remaining resources of GPUs with specified computing power in multiple working nodes,” (pages 41-42, para [n0076]).),
a total video memory capacity corresponding to the GPU (“…the hardware features in the training samples include the computing power of the GPU, the size of the GPU's video memory, and the number of GPU's computing cores… the hardware features of the GPU in the training samples are represented in a preset format, for example, as: {capability(), memory(), cores()}, where the content before the parentheses represents the GPU's computing power, the GPU's video memory size (in gigabytes (GB)), and the number of GPU's computing cores, respectively, and the content within the parentheses is the corresponding performance parameter. For example, the hardware features of a GPU can be represented as: {capability(7), memory(32), cores(2880)}.,” (pages 43– 48, para [n0078]).),
wherein service instances of one or more services run on each GPU (“in order to achieve the sharing of GPU resources among multiple container groups, a virtual unified computing device architecture (vCUDA) has been proposed to realize the sharing of GPU resources among tasks,” (pages 35-36, para [n0069]). Service instances include tasks.), and
the service instance of each service runs on one or more GPUs(“the resource utilization set includes the resource utilization of deep learning tasks on multiple specified computing power GPUs,” (pages 42-43, para [n0077]).
“Allocate GPUs and corresponding GPU resources to the deep learning task based on the remaining resources and resource requirements of multiple GPUs in the current working node,” (page 60, para [n0091]).
Service instances include tasks.); and
determining a predicted video memory capacity respectively consumed by service instances of various services (“the utilization prediction model includes a GPU computing resource utilization prediction model and a storage resource utilization prediction model... The storage resource utilization prediction model is used to predict the storage resource utilization of a deep learning task on a variety of GPUs with specified computing power, so that the GPU can allocate corresponding storage resources to it,” (pages 43– 48, para [n0078]; page 7, para [n0008]).
The predicted video memory capacity includes Sun’s predicted storage resource utilization.), (pages 43– 48, para [n0078]).
“The deep learning task and its resource requirements are sent to the selected worker node, whereby the resource requirements indicate the resource utilization of the deep learning task on a variety of specified computing power GPUs,” (page 8, para [n0010]).
“Allocate GPUs and corresponding GPU resources to the deep learning task based on the remaining resources and resource requirements of multiple GPUs in the current working node,” (page 60, para [n0091]).
Resource requirements / predicted utilization includes predicted storage resource utilization.).
Sun is not relied upon teaching obtaining, for each GPU of a plurality of GPUs, a consumed video memory capacity that has been used to run services on the GPU as well as types and a quantity of service instances running on each GPU of the plurality of GPUs.
Fan teaches obtaining, for each GPU of a plurality of GPUs, a consumed video memory capacity that has been used to run services on the GPU (“Table 3 Graphics Card Resource Management Table. On the GPU POOL, the GPU resource management table records the usage of all physical GPUs managed by all rendering hosts,” (Fan; page 51, para [n0087]- [n0089]).
“Table 4 records the detailed usage of the virtual machine's video memory and can calculate the total amount of video memory currently used, which is periodically synchronized to Table 3, the graphics card resource management table,” (Fan; pages 56-57, para [n0100]).
“…determining the used video memory of the virtual machine based on the virtual machine video memory management table; and updating the used video memory to a preset graphics card resource management table,” (Fan; page 40, para [n0064]).
Consumed video memory capacity includes Fan’s used video memory. A service instance includes a virtual machine.),
a total video memory capacity corresponding to the GPU (“sorting physical graphics cards according to their physical video memory size, with those having smaller physical video memory ranked first; and for graphics cards with the same physical video memory specifications, sorting them according to the amount of remaining video memory,” (page 39, para [n0063]).
Total video memory capacity includes Fan’s physical video memory size.), and types (“the desired graphics card performance parameters can be allocated based on the application currently running in the virtual machine, according to preset rules. The application running in the virtual machine is identified by querying a pre-defined application classification table to determine its category identifier,” (Fan; pages 46-47, para [n0076] – [n0077]).
“if the application does not exist in the preset application category table … the type of the application can be determined based on the size of the pre-loaded data package, thereby determining the type identifier of the application,” (Fan; pages 47-48para [n0079]-[n0080]).
The types can include the category identifier / type identifier of the application.) and a quantity of service instances running on each GPU of the plurality of GPUs (“GPU POOL can allocate GPU virtual resources to each VM. In other words, multiple VMs can share a single physical GPU,” (Fan; pages 5-6, para [n0007]).
“Taking the parameters in Table 3 above as an example, the expected video memory corresponding to application identifiers A, B, C, and D are 4G, 2G, 1G, and 512M, respectively,” (Fan; pages 53-54, para [n0094]).
A quantity of service instances is obtained since each application identifier / type for each applciation is recorded in the resource management table (Table 3). Service instance includes an application.)
wherein service instances of one or more services run on each GPU (“GPU POOL can allocate GPU virtual resources to each VM. In other words, multiple VMs can share a single physical GPU,” (pages 5-6, para [n0007]).), and the service instance of each service runs on one or more GPUs (“When a virtual machine starts up, the allocation device can refer to the graphics card resource management table to match a suitable rendering host and physical graphics card as the target physical graphics card for the virtual machine by using the graphics card performance configuration parameters,” (page 52, para [n0090]). A virtual machine runs on one or more GPUs. A service instance includes a virtual machine.)
and the service instance of each service runs on one or more GPUs (“Dismantle the session between virtual machine 1 and GPU POOL1, and directly allocate a graphics card to another GPU POOL that can meet the updated video memory requirements of the virtual machine,” (pages 60-61, para [n0110]). A service instance includes a virtual machine.).
Sun does not teach that the prediction of video memory capacity respectively consumed by service instances is based on the consumed video memory capacity of each GPU of the plurality of GPUs, the total video memory capacity of each GPU of the plurality of GPUs, and the types and the quantity of the service instances running on each GPU of the plurality of GPUs.
Fan teaches that the prediction of video memory capacity respectively consumed by service instances is based on the consumed video memory capacity of each GPU of the plurality of GPUs (
“Table 4 records the detailed usage of the virtual machine's video memory and can calculate the total amount of video memory currently used, which is periodically synchronized to Table 3, the graphics card resource management table,” (Fan; pages 56-57, para [n0100]).
“the allocation device stores a graphics card resource management table, as shown in Table 3. It should be noted that the graphics card resource management table is the basis for selecting the rendering host and graphics card,” (pages 50-51, para [n0086]).
“a rendering host and physical graphics card are allocated to the virtual machine according to a predetermined matching rule and an updated preset graphics card resource management table, including: sorting the updated preset graphics card resource management table according to the predetermined matching rule to obtain a sorting result; determining the expected video memory size of the virtual machine; and determining the rendering host and physical graphics card allocated to the virtual machine based on the expected video memory size and the sorting result,” (pages 38-39, para [n0062]).
“the aforementioned predetermined matching rules include, but are not limited to: sorting physical graphics cards according to their physical video memory size, with those having smaller physical video memory ranked first; and for graphics cards with the same physical video memory specifications, sorting them according to the amount of remaining video memory, with those having less remaining video memory ranked first,” (page 39, para [n0063]).
The allocation is based on the sorting result of the graphics card resource management table. Allocation is / indicates / includes a predicted memory usage. The sorting result is based on the physical video memory size and the amount of remaining video memory which may be derived from the obtained used video memory.),
the total video memory capacity of each GPU of the plurality of GPUs ((pages 38-39, para [n0062]-[n0063]). The allocation is based on the sorting result of the graphics card resource management table. Allocation is / indicates / includes a predicted memory usage. The sorting result is based on the physical video memory size. Total video memory capacity includes Fan’s physical video memory size. ) and
the types (Fan; “Taking the parameters in Table 3 above as an example, the expected video memory corresponding to application identifiers A, B, C, and D are 4G, 2G, 1G, and 512M, respectively,” (pages 53-54, para [n0094]). From the table, the application identifier / type is mapped to an expected memory size used for allocation. ) and the quantity of the service instances running on each GPU of the plurality of GPUs (Fan; “GPU POOL can allocate GPU virtual resources to each VM. In other words, multiple VMs can share a single physical GPU,” (pages 5-6, para [n0007]).
“Table 3 Graphics Card Resource Management Table. On the GPU POOL, the GPU resource management table records the usage of all physical GPUs managed by all rendering hosts,” (Fan; page 51, para [n0087] – [n0089]).
When more applications are added, each takes up memory and are recorded in the resource management table used for allocation.)
Before the effective filling date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Fan to Sun. The motivation would have been for “maximizing the utilization of video memory resources,” (page 17, para [n0026]). Additional motivation would have been to improve the efficiency of multi-tasking.
Regarding claims 12 and 20, they are rejected using the same citations and rationales described in the rejection of claim 28.
Regarding claim 31, Sun in view of Fan teaches the computer-implemented system of claim 28, wherein each service instance of the quantity of service instances runs on a virtual container (Sun; “in order to achieve the sharing of GPU resources among multiple container groups, a virtual unified computing device architecture (vCUDA) has been proposed to realize the sharing of GPU resources among tasks. However, since vCUDA adopts a simple binpack method for task scheduling and ignores the characteristics of the task itself, there is a problem of over-allocation of resources, which makes it impossible to fully utilize GPU resources.
To address the aforementioned technical issues, this application provides a GPU resource allocation method for deep learning tasks. By predicting the resource requirements of deep learning tasks, the GPU resources in the container cloud cluster are rationally allocated based on the predicted resource requirements, thereby achieving GPU resource sharing in the container cloud cluster and improving GPU utilization in the container cloud cluster,” (Sun; pages 35-37, para [n0069]- [n0070]; pages 3-4, para [n0002]-[n0003]).
Service instances include tasks.).
Regarding claims 23 and 15, they are rejected using the same citations and rationales described in the rejection of claim 31.
Regarding claim 24, Sun in view of Fan teaches the non-transitory, computer-readable medium of claim 20, comprising: writing the predicted video memory capacity (Fan; “to further track and maintain the video memory usage of the virtual machine, the 3D API Hook module in the virtual machine can intercept the functions in the 3D API used to allocate and release video memory, and record the current video memory usage (allocation /release) of the virtual machine in real time, which is recorded as the virtual machine video memory management table, as shown in Table 4,” (para [n0096]).
“Table 4 records the detailed usage of the virtual machine's video memory and can calculate the total amount of video memory currently used, which is periodically synchronized to Table 3, the graphics card resource management table,” (Fan; pages 56-57, para [n0100]).
Allocation is / indicates / includes a predicted memory usage. A predicted video memory capacity includes the memory used by a newly allocated virtual machine. When a virtual machine is newly allocated, the resource management table is updated / written.) to a service resource ledger included in a service scheduler for use by the service scheduler to allocate (Fan; “…the allocation device stores a graphics card resource management table, as shown in Table 3. It should be noted that the graphics card resource management table is the basis for selecting the rendering host and graphics card.” (pages 50-51, para [n0086]).
A service resource ledger includes the resource management table. A service scheduler includes the allocation device.), based on the service resource ledger, video memory to the service instances running on the GPUs ((Fan; pages 38-39, para [n0062]).
“When a virtual machine starts up, the allocation device can refer to the graphics card resource management table to match a suitable rendering host and physical graphics card as the target physical graphics card for the virtual machine by using the graphics card performance configuration parameters,” (Fan; page 52, para [n0090]).
A service resource ledger includes the resource management table. Allocating includes matching a rendering host and a graphics card.).
Before the effective filling date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Fan to Sun. The motivation would have been to enable faster data retrieval. Additional motivation would have been to reduce the waste of GPU resources. Additional motivation would have been to enable running more applications / services at once (Fan; para [n0007]). Additional motivation would have been for “maximizing the utilization of video memory resources,” (Fan; page 17, para [n0026]). Additional motivation would have been to improve the efficiency of multi-tasking.
Regarding claim 16, it is rejected using the same citations and rationales described in the rejection of claim 24.
Regarding claim 25, Sun in view of Fan teaches the non-transitory, computer-readable medium of claim 20, wherein: the service instances of one or more services run on each GPU (Fan; “GPU POOL centrally manages the resources of multiple physical GPUs. GPU POOL can allocate GPU virtual resources to each VM. In other words, multiple VMs can share a single physical GPU,” (pages 5-6, para [n0007]).); service instances of one or more services in a target service type set run on each GPU (Fan; “On the GPU POOL, the GPU resource management table records the usage of all physical GPUs managed by all rendering hosts,” (page 51, para [n0089]).
“the allocation device stores a graphics card resource management table, as shown in Table 3,” (Fan; pages 50-51, para [n0086]).
“Taking the parameters in Table 3 above as an example, the expected video memory corresponding to application identifiers A, B, C, and D are 4G, 2G, 1G, and 512M, respectively,” (Fan; pages 53-54, para [n0094]).
“The application running in the virtual machine is identified by querying a pre-defined application classification table to determine its category identifier,” (Fan; pages 46-47, para [n0076]). An application / virtual machine corresponds to an application / category identifier.
The resource management table (Table 3) is on the GPU pool. The target service type set includes the applications / virtual machines and / or their corresponding application identifiers present in a given GPU pool.); and
determining a predicted video memory capacity respectively consumed by service instances of various services, comprises: determining a predicted video memory capacity respectively consumed by service instances of various services in the target service type set (Sun; “the utilization prediction model includes a GPU computing resource utilization prediction model and a storage resource utilization prediction model... The storage resource utilization prediction model is used to predict the storage resource utilization of a deep learning task on a variety of GPUs with specified computing power, so that the GPU can allocate corresponding storage resources to it,” (pages 43– 48, para [n0078]; page 7, para [n0008]).
The predicted video memory capacity includes Sun’s predicted storage resource utilization.
Fan; “Taking the parameters in Table 3 above as an example, the expected video memory corresponding to application identifiers A, B, C, and D are 4G, 2G, 1G, and 512M, respectively,” (pages 53-54, para [n0094]).
“…record the current video memory usage (allocation /release) of the virtual machine in real time, which is recorded as the virtual machine video memory management table, as shown in Table 4,” (Fan; para [n0096]).
“Table 4 records the detailed usage of the virtual machine's video memory and can calculate the total amount of video memory currently used, which is periodically synchronized to Table 3, the graphics card resource management table,” (Fan; para [n0100]).
The resource management table (Table 3) is on the GPU pool. The target service type set includes the application identifiers present in a given GPU pool. After combination, updating the resource management table includes updating / determining predicted/ expected / allocated video memory for application identifiers a given GPU pool.).
Before the effective filling date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Fan to Sun. The motivation would have been to reduce the waste of GPU resources. Additional motivation would have been to reduce the waste of GPU resource in the case that different applications / services have different requirements for graphics card video memory (Fan; para [n0013]). Additional motivation would have been to enable running more applications / services at once (Fan; para [n0007]). Additional motivation would have been for “maximizing the utilization of video memory resources,” (Fan; page 17, para [n0026]). Additional motivation would have been to improve the efficiency of multi-tasking.
Regarding claim 17, it is rejected using the same citations and rationales described in the rejection of claim 25.
Regarding claim 26, Sun in view of Fan teaches the non-transitory, computer-readable medium of claim 25, wherein obtaining, for each of a plurality of GPUs, a consumed video memory capacity that has been used to run services on the GPU, and a total video memory capacity corresponding to the GPU, comprises: obtaining, for each of a plurality of GPUs, a consumed video memory capacity that has been used to run services on the GPU, and a total video memory capacity corresponding to the GPU, in response to a change in the target service type set (Fan; “to further track and maintain the video memory usage of the virtual machine, the 3D API Hook module in the virtual machine can intercept the functions in the 3D API used to allocate and release video memory, and record the current video memory usage (allocation /release) of the virtual machine in real time, which is recorded as the virtual machine video memory management table, as shown in Table 4,” (para [n0096]).
“Table 4 records the detailed usage of the virtual machine's video memory and can calculate the total amount of video memory currently used, which is periodically synchronized to Table 3, the graphics card resource management table,” (Fan; para [n0100]).
The allocation / release of video memory is a change in the applications / virtual machines and / or their corresponding application identifiers present in a given GPU pool. Updating the resource management table in real-time in response to the allocation / release of video memory includes obtaining / determining consumed video memory in response to a change in the target service type set.).
Before the effective filling date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Fan to Sun. The motivation would have been to reduce the waste of GPU resources. Additional motivation would have been to prevent “display problems such as stuttering,” and improve user experience (Fan; para [n0103]). Additional motivation would have been to enable running more applications / services at once (Fan; para [n0007]). Additional motivation would have been to improve performance and / or efficiency. Additional motivation would have been for “maximizing the utilization of video memory resources,” (Fan; page 17, para [n0026]).
Regarding claim 18, it is rejected using the same citations and rationales described in the rejection of claim 26.
Regarding claim 27, Sun in view of Fan teaches the non-transitory, computer-readable medium of claim 26, wherein the change in the target service type set comprises: adding or removing a service type to or from the target service type set (Fan; “Dismantle the session between virtual machine 1 and GPU POOL1, and directly allocate a graphics card to another GPU POOL that can meet the updated video memory requirements of the virtual machine,” (page 60-61, para [n0110]).
“the session between the target virtual machine and the rendering host is torn down, and the preset graphics card resource management table is updated,” (Fan; pages 35-36, para [n0059]).
Removing a service type from a target service type set includes dismantling / tearing down the session between a virtual machine and a GPU pool.).
Before the effective filling date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Fan to Sun. The motivation would have been to more efficiently reuse memory. Additional motivation would have been to reduce the waste of GPU resource. Additional motivation would have been to enable running more applications / services at once (Fan; para [n0007]). Additional motivation would have been to improve performance and / or efficiency. Additional motivation would have been for “maximizing the utilization of video memory resources,” (Fan; page 17, para [n0026]).
Regarding claim 19, it is rejected using the same citations and rationales described in the rejection of claim 27.
Allowable Subject Matter
Claims 13, 21, and 29 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Claims 14, 22, and 30 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims. Claims 14, 22, and 30 are in allowable form due to their dependency on claims 13, 21, and 29.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ERICA G THERKORN whose telephone number is (571)272-2939. The examiner can normally be reached Monday - Friday 9:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona Faulk can be reached at 571-272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ERICA G THERKORN/Examiner, Art Unit 2618
/DEVONA E FAULK/Supervisory Patent Examiner, Art Unit 2618
Prosecution Insights