DETAILED ACTION
This office action is in response to claims filed 11 June 2026
Claims 1-9, and 12-34 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Neither applicant’s amendments, nor applicant’s remarks address the rejections made under 35 USC § 112b, and therefore the rejections have been maintained.
Applicant’s arguments with respect to the rejection made under 35 USC § 103 have been considered but are moot because either the arguments do not specifically challenge the new references (SLUPIK) applied in the prior rejection of record, or the arguments are applied to claims which are found to be allowable over the prior art.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) in claims 21, 27, and 28 are:
“An artificial intelligence (AI) interface service…configured to execute”;
“A scheduler…configured to identify…identify…calculate…select…assign…execute”;
“The scheduler is further configured to schedule”; and
“The scheduler is further configured to schedule”.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. In [0047] Artificial intelligence interface services and Schedulers are run “via processors and memory” implying they are software stored in memory and executed by processors.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Allowable Subject Matter
Claims 9, 12-20, 30, and 33 are allowed.
Claim Objections
Claims 1, and 21 are objected to because of the following informalities: The acronyms “GPU” and CPU” are not explained. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 21-28, 31, and 34 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 21,
On lines 24-25, the claim fails to particularly point out or distinctly claim whether the “artificial intelligence (AI) inference service…[is] configured to execute the machine learning model” (lines 7-8) or the “scheduler, communicatively coupled to the AI inference service [is] configured to…execute the machine learning model” (lines 9, and 24-25) executes the machine learning module. In other words, the claim removed clarifying language and now states that the AI inference service, AND the scheduler execute the machine learning module, and therefore, the claim does not particularly point out or distinctly claim which one executes the machine learning model. For examination purposes, the examiner will interpret the machine learning model as being executed by the AI inference service.
Regarding claims 22-28, 31, and 34, they are dependent upon rejected claim 21, and fail to resolve the deficiencies thereof. They are therefore rejected for similar rationale.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 7-8, and 32 are rejected under 35 U.S.C. 103 as being unpatentable over METSCH et al. Pub. No.: US 2019/0324799 A1 (hereafter METSCH), in view of ROSS et al. Patent No.: US 10,685,295 B1 (hereafter Ross), in view of CAO et al. Patent No.: US 10,827,020 B1 (hereafter CAO), in view of SLUPIK et al. Pub. No.: US 2016/0119572 A1 (hereafter SLUPIK).
METSCH, ROSS, and CAO were cited previously.
Regarding claim 1, METSCH teaches the invention substantially as claimed, including:
At least one non-transitory computer readable medium encoded with instructions which, when executed, cause a system to perform actions ([0048] A non-transitory machine readable storage medium including program code, when executed, to cause a programmable processor to perform the method described above or below) comprising:
receiving a request to execute a [computational task] ([0002] A user can request from the computing service to solve a computational task);
identifying one or more nodes of a clustered computing system having a plurality of nodes…each of the one or more nodes comprising a plurality of candidate hardware resources for executing the [computational task] ([0067] FIG. 7 shows an example of predicting 700 workload characteristics. Predicting workload characteristics may enable to distribute workloads on a system such that resources of the system are used more equally and/or a utilization level of resources of the system is more uniform. When receiving 702 an orchestration event, predicting 700 may be performed in a foreground flow during provisioning of a workload. Out of all possible (or shortlisted) resource aggregates/compute hosts (i.e., “nodes”) in a cluster/system those are picked or determined 704, which satisfy the resource request requirements);
identifying one or more candidate hardware resources from the plurality of candidate hardware resources ([0068] For each candidate (or determined shared resource), the involved resources (respectively subsystems) may be identified and the models as stored in the database (see e.g. the examples according to FIG. 4 and FIG. 5) can be loaded 706. The models may be provided 708 by the database. Fingerprints of the workload that is to be handled, and the fingerprint describing the current behavior of the resource may be used, and a future fingerprint may be predicted 710);
calculating a score for each of the identified one or more candidate hardware resources ([0069] The prediction can be used to calculate 712 a utility (i.e. utility “score”, as referenced in [0072] cited below) of a resource) based at least on execution priorities for the [computational process] ([0069] This utility calculation can be based on the dominant state (e.g. its expected level of utilization/saturation/throughput/latency etc.) that the resource will reside in once workload(s) to be handled are place on it, as well as the probability/frequency to stay in the state. [0070] The factor C can be used to positively or negatively reward the fact that the resource is for example not in a single high utilization state. Factor C can be a fixed value, or a term based on the probability or the frequency of a state transition (i.e., factor C rewards, or “prioritizes” a predicted execution state of the resource while executing a computational process));
selecting one of the identified candidate hardware resources based at least on the calculating ([0071] The Utility of a resource can then be used to pick (or select 130) the best candidate (or shared resource, respectively). Furthermore other utilities about the resources behavior (e.g. for preventing interference) can be included. [0072] These individual utility scores can be comparable between the sub-systems/resources of a resource aggregate (e.g. a compute hosts) and therefore may allow for fast reasoning);
assigning the [computational process] to the selected identified candidate hardware resource; and executing the [computational process] using the selected identified candidate hardware resource ([0023] Further, the predicted workload characteristics may be provided and selecting 130 one of the at least two determined shared resources is based on the predicted workload characteristics according to the method 100. In some examples, the predicted workload characteristics of the determined shared resources differ from each other, for example if another computational process is already running on one of the determined shared resources and/or the determined shared resources comprise different subsystems, e.g. hardware components. The differing workload characteristics can be used to select 130 the one shared resource to perform the computational process.).
While METSCH discusses selecting and allocating of nodes and corresponding resources to computational processes for execution, METSCH does not explicitly teach that these processes are
machine learning models;
However, in analogous art that similarly teaches allocation of resources, ROSS teaches computational processes as:
machine learning models ([Column 3, Line 66-Column 4, Line 1] The processing system (100) receives a machine learning model to be executed on a special purpose machine learning processor (202). [Column 5, Lines 18-21] The processing system (100) allocates resources of the special purpose machine learning model processor based on the determined amount of resources required by the executable binary (212));
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to have combined ROSS’s teaching of a machine learning model as a computational process to execute using allocated resources, with METSCH’s teaching of allocating resources to a computational process based on node and resource evaluations, to realize, with a reasonable expectation of success, a system that allocates resources to a computational process based on node and resource evaluations, as in METSCH, where the computational processes are machine learning models, as in ROSS. A person having ordinary skill would have been motivated to make this combination to maximize performance and efficiency of a machine learning model task by choosing the optimal combination of machine learning models that will run the machine learning task (ROSS Column 8, Lines 32-37).
While METSCH and ROSS discuss identification of particular nodes in a clustered computing system that satisfy resource request requirements, METSCH and ROSS do not explicitly teach:
identifying one or more nodes of a clustered computing system having a plurality of nodes based at least in part on node affinity of the one or more nodes.
However, in analogous art that similarly allocates computational processes to nodes of a clustered computing system, CAO teaches:
identifying one or more nodes of a clustered computing system having a plurality of nodes based at least in part on node affinity of the one or more nodes ([Column 6, Line 57-Column 7, Line 2] Scheduler extension 230 may further include a configuration executor 236. Configuration executor 236 may execute the assignments of microservices to cluster nodes as determined by microservice allocator 234…In an example implementation, the container orchestration platform (not shown) may include node affinity data to guide the container orchestration platform to place the microservices in the correspondingly assigned cluster nodes (i.e., cluster node identification for selection is based at least partially on “node affinity data”)).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to have combined CAO’s teaching of identifying nodes for selection based on node affinity, with the combination of METSCH and ROSS’s teaching of identifying nodes for use in executing machine learning models that satisfy resource request requirements, to realize, with a reasonable expectation of success, a system that selects nodes for use in executing machine learning models that satisfy resource request requirements, as in METSCH and ROSS, where the requirements include a node affinity requirement, as in CAO. A person having ordinary skill would have been motivated to make this combination to make more efficient workload placement decisions (CAO Column 2, Lines 13-15).
While METSCH, ROSS, and CAO discuss identifying nodes to run AI workloads, they do not explicitly teach:
analyzing types of the one or more candidate hardware resources in a particular order including
analyzing GPUs of the one or more nodes in the plurality of nodes for capability to run the machine learning model, and
after analyzing the GPUs, analyzing CPUs of the one or more nodes in the plurality of nodes for capability to run the machine learning model, wherein each of the identified one or more candidate hardware resources comprises at least one of the analyzed CPUs.
However, in analogous art that similarly discusses identifying resources to run workloads, SLUPIK teaches:
analyzing types of the one or more candidate hardware resources in a particular order including analyzing GPUs of the one or more nodes in the plurality of nodes for capability to run the [workload], and after analyzing the GPUs, analyzing CPUs of the one or more nodes in the plurality of nodes for capability to run the [workload], wherein each of the identified one or more candidate hardware resources comprises at least one of the analyzed CPUs ([0070] The first of these consists of detecting that a recoverable error occurred in a given resource, such as a hardware decoder. Such detection can be provided in the form of software or API-based reporting capabilities of the resource itself, or via an external observation or inquiry made of a resource. The second step consists of substituting a failed or failing resource (such as a GPU-based hardware-implemented decoder) with a more robust and operational resource (such as a CPU-based software-implemented decoder) available within the resource pool 500. It will be appreciated that such a substitution should be carefully carried out so as to minimally impair the operation of any other aspect of implementations described herein (i.e., GPU analysis detects a recoverable error and substituting a CPU capable of performing the workload)).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to have combined SLUPIK’s teaching of analyzing GPUs for failure when executing workloads, and further substituting a CPU to execute the workload, with the combination of METSCH, ROSS, and CAO’s teaching of executing machine learning model workloads on resources, to realize, with a reasonable expectation of success, a system that executes machine learning model workloads, as in METSCH, ROSS, and CAO, by analyzing a GPU, determining that a failure has occurred, and transferring execution to a CPU, as in SLUPIK. A person having ordinary skill would have been motivated to make this combination to better recover from and handle system crashes (SLUPIK [0070]).
Regarding claim 2, METSCH further teaches:
wherein identifying the one or more candidate hardware resources is further based at least in part on analyzing compute resource metrics of the [computational process] ([0068] For each candidate (or determined shared resource), the involved resources (respectively subsystems) may be identified and the models as stored in the database (see e.g. the examples according to FIG. 4 and FIG. 5) can be loaded 706. The models may be provided 708 by the database. Fingerprints of the workload that is to be handled, and the fingerprint describing the current behavior of the resource (i.e., resource fingerprints represent analysis of metrics related to the resources) may be used, and a future fingerprint may be predicted 710) to determine an approximate compute resource need to run the machine learning model ([0069] This utility calculation can be based on the dominant state (e.g. its expected level of utilization/saturation/throughput/latency etc.) that the resource will reside in once workload(s) to be handled are place on it, as well as the probability/frequency to stay in the state (i.e., expected utilization/saturation/throughput/latency etc. represents the approximate usage or “need” of the resource in question)).
Regarding claim 3, ROSS further teaches:
the actions further comprising: analyzing the compute resource metrics based at least in part on analyzing a deep learning model graph structure including what operations are performed on each graph node of a plurality of graph nodes ([Column 6, Lines 50-61] Another benefit to knowing the amount of resources necessary to run a machine learning model is improving data center efficiency. As described above, an example system can determine the amount of resources necessary to run each machine learning model (i.e., “compute resource metrics”). Special purpose machine learning model processors can be tasked with a specific number of machine learning model operations. Thus the number of operations, the amount of IO, and the amount of storage required to execute the operations of computational dataflow graphs representing machine learning models (i.e., “deep learning model graph structure”) that are assigned to execute in the datacenter may be known with a high degree of precision (i.e., operations executed on resources of a computational dataflow graph represent “operations performed on graph nodes”)).
Regarding claim 7, ROSS further teaches:
wherein the assigning is further based at least on compute resource metrics for the machine learning model ([Column 5, Lines 18-21] The processing system (100) allocates resources of the special purpose machine learning model processor based on the determined amount of resources required by the executable binary (212) (i.e., determined amount of resources required by the executable binary is indicative of “computer resource metrics” required by the compiled “machine learning model” as discussed in at least Column 1, Lines 28-38)).
Regarding claim 8, METSCH further teaches:
the clustered computing system comprises a multi-node edge (Out of all possible (or shortlisted) resource aggregates/compute hosts in a cluster/system those are picked or determined 704, which satisfy the resource request requirements (i.e., a cluster comprises multiple host nodes, and is therefore considered a “multi-node edge”)).
Regarding claim 32, METSCH further teaches:
wherein identifying the one or more candidate hardware resources is further based at least in part on analyzing the one or more candidate hardware resources in a particular order ([0068] For each candidate (or determined shared resource), the involved resources (respectively subsystems) may be identified and the models as stored in the database (see e.g. the examples according to FIG. 4 and FIG. 5) can be loaded 706. The models may be provided 708 by the database. Fingerprints of the workload that is to be handled, and the fingerprint describing the current behavior of the resource may be used, and a future fingerprint may be predicted 710 (i.e., each resource of the candidate nodes is identified in some “order”)).
Claims 5-6 are rejected under 35 U.S.C. 103 as being unpatentable over METSCH, in view of ROSS, in view of CAO, in view of SLUPIK as cited in claims 1, and 9 above, and in further view of ZHAO et al. Pub. No.: US 2019/0384641 A1 (hereafter ZHAO).
ZHAO was cited previously.
Regarding claim 5, while METSCH, ROSS, CAO, and SLUPIK discuss calculating scores for node resources, they do not explicitly teach:
wherein the calculating is further based at least on a weighted sum of execution priorities for each of the identified one or more candidate hardware resources
However, in analogous art that similarly teaches scoring of resources in nodes for allocation, ZHAO teaches:
wherein the calculating is further based at least on a weighted sum of execution priorities for each of the identified one or more candidate hardware resources ([0037] At block 440, multiple computing resources (i.e., “identified candidate hardware resources”) are ranked based on status information 330 (i.e., “scores”) of the multiple computing resources so as to obtain a resource list 332. The status information 330 here may involve indicators of various aspects of the computing resources 160. According to example embodiments of the present disclosure, the status information of the multiple computing resources 160 comprises at least any one indicator of processing capacity information, memory resource information and bandwidth resource information of the multiple computing resources 160. [0044] In Equation 1, Status (i) represents status information of the i.sup.th computing resource in the resource pool 320, ProcessingCapacity represents processing capacity information, Weight.sub.processing capacity represents importance of the processing capacity information, MemoryCapacity represents memory resource information, Weight.sub.memory capacity represents importance of the memory resource information, BandWidth represents bandwidth resource information, and Weight.sub.bandwidth represents importance of the bandwidth resource information (i.e., equation 1 sums the weighted values for each resource)).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to have combined ZHAO’s teaching of using a weighted sum of values to determine a score for resources, with the combination of METSCH, ROSS, CAO, and SLUPIK’s teaching of determining scores for resources, to realize, with a reasonable expectation of success, a system that determines scores for resources, as in METSCH, ROSS, CAO, and SLUPIK based on a weighted sum, as in ZHAO. A person having ordinary skill would have been motivated to make this combination to enable a user or system to adapt resource scores to better achieve allocation objectives.
Regarding claim 6, ZHAO further teaches:
the execution priorities for the machine learning model comprise per-node per-time period inference request counts, per-node number of kubernetes (k8) pods running, per-node machine learning models running, free processor memory space, processor utilization, or combinations thereof ([0037] At block 440, multiple computing resources are ranked (i.e., “scored”) based on status information 330 of the multiple computing resources so as to obtain a resource list 332. The status information 330 here may involve indicators of various aspects of the computing resources 160. According to example embodiments of the present disclosure, the status information of the multiple computing resources 160 comprises at least any one indicator of processing capacity information, memory resource information (i.e., the priority placed on resources by the machine learning model include at least “free processor memory space” and “processor utilization”) and bandwidth resource information of the multiple computing resources 160 (i.e., [0054] also describes how status information is indicative of % of resource utilization, including processor utilization or presumably memory space utilization)).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over METSCH, in view of ROSS, in view of CAO, in view of SLUPIK, as applied to claims 1, above, and in further view of NAMBIAR et al. Pub. No.: US 2016/0234071 A1 (hereafter NAMBIAR).
NAMBIAR was cited previously.
Regarding claim 4, while the combination of METSCH, ROSS, CAO, and SLUPIK teaches identifying candidate resources for executing a machine learning model based on node affinity, the combination of METSCH, ROSS, CAO, and SLUPIK does not explicitly teach:
wherein the identifying is further based at least in part on rack-awareness in the clustered computing system.
However, in analogous art that similarly identifies resources based on affinities, NAMBIAR teaches:
wherein the identifying is further based at least in part on rack-awareness in the clustered computing system ([0032] In various embodiments, when application scheduler 32 receives a request to execute a job within distributed application 30, application scheduler 32 determines what resources are available for executing the requested job, including what resources are available for placing (storing) data associated with the workloads. Application scheduler 32 can determine where to place data among hosts 16 using a data placement policy 34, along with a scheduling policy. Data placement policy 34 can specify guidelines for selecting network nodes (such as hosts 16), including but not limited to, node availability, node capacity, node locality, data placement cost, data transfer costs, network topology, user preferences associated with the storage node, and/or other data placement guideline…data placement policy 34 and/or replica placement policy 36 may define a rack awareness policy that specifies that data should be placed on network nodes associated with different racks (i.e., rack awareness policy also specifies the affinity/anti-affinity data has with particular nodes on particular racks)).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to have combined NAMBIAR’s teaching of selecting resources to execute a task based on rack awareness policy that specifies affinity between nodes and racks, with the combination of METSCH, ROSS, CAO, and SLUPIK’s teaching of selecting resources to execute a machine learning model based on a node affinity, to realize, with a reasonable expectation of success, a system that selects resources based at least on a rack awareness policy, as in NAMBIAR, and a node affinity that is used to execute a machine learning model, as in METSCH, ROSS, CAO, and SLUPIK. A person having ordinary skill would have been motivated to make this combination to optimize deployment of distributed application frameworks while enhancing network performance associated with using the frameworks (NAMBIAR [0002]).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL W AYERS whose telephone number is (571)272-6420. The examiner can normally be reached M-F 8:30-5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Aimee Li can be reached at (571) 272-4169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL W AYERS/Primary Examiner, Art Unit 2195
Prosecution Insights