DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is responsive to the Application filed on 3/05/2024. Claims 1-21 are pending in the case. Claims 1, 10, and 13 are independent claims.
Specification Objections
The specification is objected to because of the typographical error on page 1, line 9, which reads: “the cost to use hardware to operate aspects of model, or other resources available to rum the model.” (Emphasis added). It is believed that “rum” should be “run.”
Claim Rejections - 35 U.S.C. § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to non-statutory subject matter. During examination, the claims must be interpreted as broadly as their terms reasonably allow. In re American Academy of Science Tech Center, 367 F.3d 1359, 1369, 70 U.S.P.Q.2d 1827, 1834 (Fed. Cir. 2004). Independent claim 1 recites a “computer readable media,” which is not comprehensively defined by the specification. The broadest reasonable interpretation of a claim drawn to a computer readable medium covers forms of transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media, particularly when the specification is silent. Transitory propagating signals are non-statutory subject matter. In re Nuijten, 500 F.3d 1346, 1356-57, 84 U.S.P.Q.2d 1495, 1502 (Fed. Cir. 2007) (transitory embodiments are not directed to statutory subject matter). See also Subject Matter Eligibility of Computer Readable Media, 1351 Off. Gaz. Pat. Office 212 (Feb. 23, 2010). Examiner suggests adding the word “non-transitory.”
Claims 1-21 are also rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claims 1-9 are directed towards the statutory category of an article of manufacture. Claims 10-21 are directed towards the statutory category of a machine.
With respect to claim 1:
2A Prong 1: This claim is directed to a judicial exception.
… the method comprising:… determining which resources of the two or more resources are to be provisioned for operation of the machine learning model based on the one or more resource constraints and the indication of two or more resources (mental process).
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
One or more instances of computer-readable media collectively having contents configured to cause a computing device to perform a method for provisioning machine learning model resources (merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f));
receiving an indication of a machine learning model (adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP § 2106.05(g));
receiving an indication of one or more resource constraints (adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP § 2106.05(g));
receiving an indication of two or more resources available for operation of the machine learning model (adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP § 2106.05(g)); and
causing the determined resources to be provisioned for operation of the machine learning model (merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f)).
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
One or more instances of computer-readable media collectively having contents configured to cause a computing device to perform a method for provisioning machine learning model resources (merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f));
receiving an indication of a machine learning model (MPEP 2106.05(d) indicates that merely “storing and retrieving information in memory” and/or "receiving or transmitting data over a network" are well‐understood, routine, conventional functions when they are claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed step is well-understood, routine, conventional activity is supported under Berkheimer);
receiving an indication of one or more resource constraints (MPEP 2106.05(d) indicates that merely “storing and retrieving information in memory” and/or "receiving or transmitting data over a network" are well‐understood, routine, conventional functions when they are claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed step is well-understood, routine, conventional activity is supported under Berkheimer);
receiving an indication of two or more resources available for operation of the machine learning model (MPEP 2106.05(d) indicates that merely “storing and retrieving information in memory” and/or "receiving or transmitting data over a network" are well‐understood, routine, conventional functions when they are claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed step is well-understood, routine, conventional activity is supported under Berkheimer); and
causing the determined resources to be provisioned for operation of the machine learning model (merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f)).
With respect to claim 2:
2A Prong 1: This claim is directed to a judicial exception.
determining whether the machine learning model is optimized for operation with at least a portion of the determined resources (mental process); and
based on the determining, optimizing the machine learning model for operation with the portion of the determined resources (mental process).
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 3:
2A Prong 1: This claim is directed to a judicial exception.
determining which new resources of the two or more changed resources are to be provisioned for operation of the machine learning model based on the or more resource constraints and the indication of the changed two or more resources (mental process).
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
receiving an indication that at least a portion of the two or more resources available for operation of the machine learning model have changed (adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP § 2106.05(g)); and
causing the new resources to be provisioned for operation of the machine learning model (merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f)).
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
receiving an indication that at least a portion of the two or more resources available for operation of the machine learning model have changed (MPEP 2106.05(d) indicates that merely “storing and retrieving information in memory” and/or "receiving or transmitting data over a network" are well‐understood, routine, conventional functions when they are claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed step is well-understood, routine, conventional activity is supported under Berkheimer); and
causing the new resources to be provisioned for operation of the machine learning model (merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f)).
With respect to claim 4:
2A Prong 1: This claim is directed to a judicial exception.
optimizing the machine learning model for operation with the new resources (mental process).
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 5:
2A Prong 1: This claim is directed to a judicial exception.
an indication that at least a portion of the one or more resource constraints have changed (mental process);
an indication that software used to determine which resources are to be provisioned for operation of the machine learning model have changed (mental process); and
an indication that at least one aspect of the two or more resources available for operation of the machine learning model have changed (mental process).
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 6:
2A Prong 1: This claim is directed to a judicial exception.
for each respective resource of the two or more resources: determining whether inputs to the machine learning model can be batched while the respective resource is being used for operation of the machine learning model based on the machine learning model, the respective resource, and the resource constraints (mental process); and
determining which resources of the two or more resources are to be provisioned for operation of the machine learning model based on the one or more resource constraints, the indication of two or more resources, and the determinations of whether inputs to the machine learning model can be batched (mental process).
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 7:
2A Prong 1: This claim is directed to a judicial exception.
identifying a plurality of nodes based on the determined resources, each node including at least a portion of the determined resources (mental process).
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
generating a plurality of model containers based on the determined resources and the plurality of nodes, each model container including at least a portion of the machine learning model (merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f)); and
for each respective node of the plurality of nodes: causing at least one model container of the plurality of model containers to be deployed to the respective node (merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f)).
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
generating a plurality of model containers based on the determined resources and the plurality of nodes, each model container including at least a portion of the machine learning model (merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f)); and
for each respective node of the plurality of nodes: causing at least one model container of the plurality of model containers to be deployed to the respective node (merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f)).
With respect to claim 8:
2A Prong 1: This claim is directed to a judicial exception.
for each respective resource of the two or more resources: determining whether the extent to which the operation of the machine learning model utilizes the respective resource based on the machine learning model and the resource constraints (mental process);
based on the determined extent to which the machine learning model utilizes the respective resource, determining whether the multiple model containers are able to use the respective resource at the same time (mental process); and
determining which resources of the two or more resources are to be provisioned for operation of the machine learning model based on the one or more resource constraints, the indication of two or more resources, and the determinations of whether multiple model containers are able to use each resource of the two or more resources (mental process).
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 9:
2A Prong 1: This claim is directed to a judicial exception.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
causing operation of the machine learning model with the provisioned resources (merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f)).
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
causing operation of the machine learning model with the provisioned resources (merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f)).
With respect to claim 10:
2A Prong 1: This claim is directed to a judicial exception.
One or more storage devices collectively storing a machine learning model resource provisioning data structure, the data structure comprising (mental process – paper and pencil);
information specifying a machine learning model (mental process);
information specifying one or more resource constraints (mental process); and
information specifying two or more resources available for operation of the machine learning model, such that the information specifying the one or more resource constraints and the information specifying the two or more resources are usable to determine which resources of the two or more resources are to be provisioned for operation of the machine learning model (mental process).
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 11:
2A Prong 1: This claim is directed to a judicial exception.
information specifying a plurality of model containers, each model container including at least a portion of the machine learning model (mental process); and
such that the information specifying the plurality of model containers is usable to cause the determined resources to be provisioned for operation of the machine learning model (mental process);
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 12:
2A Prong 1: This claim is directed to a judicial exception.
information specifying a plurality of instances of the model, each instance of the model being optimized for operation using a set of resources (mental process).
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The remaining claims 13-21 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more for at least the same reasons as those given above with respect to claims 1-12 with only the addition of generic computer components under step 2A prong 1. Under the broadest reasonable interpretation, these limitations are process steps that cover mental processes including an observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper but for the recitation of a generic computer component. If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the "Mental Process" grouping of abstract ideas. A person would readily be able to perform this process either mentally or with the assistance of pen and paper. See MPEP § 2106.04(a)(2). Limitations that merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f). These additional elements do not integrate the judicial exception into a practical application under step 2A prong 2. Refer to MPEP §2106.04(d). Moreover, the limitations are merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f). These additional elements do not recite any additional elements/limitations that amount to significantly more. Accordingly, the claimed invention recites an abstract idea without significantly more.
Claim Rejections - 35 U.S.C. § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. §§ 102 and 103 (or as subject to pre-AIA 35 U.S.C. §§ 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant are advised of the obligation under 37 C.F.R. § 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention.
Claims 1, 3, 5, 7-10, 13, 14, 16, 18, 20, and 21 are rejected under 35 U.S.C. § 103 as being unpatentable over Nasr-Azadani et al. (U.S. Pat. App. Pub. No. 2022/0012089, hereinafter Nasr-Azadani) in view of Poothia et al. (U.S. Pat. App. Pub. No. 2022/0083389, hereinafter Poothia).
As to independent claim 1, Nasr-Azadani teaches:
One or more instances of computer-readable media collectively having contents configured to cause a computing device to perform a method for provisioning machine learning model resources, the method comprising (Paragraph 63):
receiving an indication of a machine learning model (Claim 1, "receive input data, the input data comprising platform-independent task parameters and target metrics associated with a user task." Claim 2, "the user task comprises a machine learning model");
receiving an indication of one or more resource constraints (Paragraph 35, "specified target metrics or constraints (hardware usage, cost, time constraints, accuracy, and the like)");…
causing the determined resources to be provisioned for operation of the machine learning model (Claim 1, "automatically generate a computing architecture and computing platform selection and computing resource prediction based on the input data and a hardware profile of the computer cluster using a computing resource prediction engine; automatically map the user task to one or more of the set of computing platforms and to a scheduling action among a set of scheduling actions using a trained intelligent computing resource assignment engine; automatically schedule the user task according to the scheduling action").
Nasr-Azadani does not appear to expressly teach receiving an indication of two or more resources available for operation of the machine learning model; and determining which resources of the two or more resources are to be provisioned for operation of the machine learning model based on the one or more resource constraints and the indication of two or more resources.
Poothia teaches receiving an indication of two or more resources available for operation of the machine learning model (Paragraph 12, "identify candidate hardware resources, from a plurality of hardware resources, capable of executing the machine learning model"); and determining which resources of the two or more resources are to be provisioned for operation of the machine learning model based on the one or more resource constraints and the indication of two or more resources (Paragraph 12, "To select which hardware resource from the identified candidate hardware resources on which to schedule the machine learning model, AI inference service may, using a scoring algorithm, calculate (e.g., determine) a score for each identified candidate hardware resource").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the resource prediction and workload provisioning of Nasr-Azadani to include the resource scheduling of Poothia to determine which node and which hardware accelerator on which to schedule a ML model to run efficiently (see Poothia at paragraph 4).
As to dependent claim 3, Nasr-Azadani further teaches the method for provisioning machine learning model resources further comprises: receiving an indication that at least a portion of the two or more resources available for operation of the machine learning model have changed (Paragraph 40, "A telemetry Application Programming Interface (API) 470 associated with the HPC cluster 450 may be further provided for monitoring the status of the HPC cluster 450 with respect to, for example, hardware operating and usage status of the HPC cluster 450, hardware/software license consumption status, network and storage status, and status of the user tasks and jobs, as shown in FIG. 4"); determining which new resources of the two or more changed resources are to be provisioned for operation of the machine learning model based on the or more resource constraints and the indication of the changed two or more resources (Paragraph 40, "Such information may be fed back via the telemetry APIs to the intelligent job/task assignment engine 430 for dynamic and intelligent adjustment of resource allocation, user task re-routing, user task termination, and other actions"); and causing the new resources to be provisioned for operation of the machine learning model (Paragraph 40, "adjustment of resource allocation, user task re-routing, user task termination, and other actions").
As to dependent claim 5, Nasr-Azadani further teaches the indication that at least a portion of the two or more resources have changed comprises one or more of: an indication that at least a portion of the one or more resource constraints have changed; an indication that software used to determine which resources are to be provisioned for operation of the machine learning model have changed; and an indication that at least one aspect of the two or more resources available for operation of the machine learning model have changed (Paragraph 40, "A telemetry Application Programming Interface (API) 470 associated with the HPC cluster 450 may be further provided for monitoring the status of the HPC cluster 450 with respect to, for example, hardware operating and usage status of the HPC cluster 450, hardware/software license consumption status, network and storage status, and status of the user tasks and jobs, as shown in FIG. 4." Paragraph 60, "The intelligent job/task assignment function may further consider unpredicted uncertainties 840 and may be based on the assignment policies 850. The uncertainties for example, may include unpredicted hardware failures and inaccurate resource predictions by the resource prediction twin." Paragraph 16, "such as early termination of running jobs which frees up previously occupied computing resources").
As to dependent claim 7,
Nasr-Azadani further teaches for each respective node of the plurality of nodes: causing at least one model container of the plurality of model containers to be deployed to the respective node (Paragraph 31, "deployed user AI model").
Nasr-Azadani does not appear to expressly teach determining which resources of the two or more resources are to be provisioned for operation of the machine learning model further comprises: identifying a plurality of nodes based on the determined resources, each node including at least a portion of the determined resources; generating a plurality of model containers based on the determined resources and the plurality of nodes, each model container including at least a portion of the machine learning model.
Poothia further teaches determining which resources of the two or more resources are to be provisioned for operation of the machine learning model further comprises: identifying a plurality of nodes based on the determined resources, each node including at least a portion of the determined resources (Claim 5, "calculating based at least on a weighted sum of the execution priorities for each of the identified candidate hardware resources"); generating a plurality of model containers based on the determined resources and the plurality of nodes, each model container including at least a portion of the machine learning model (Claim 6, "the execution priorities comprise per-node per-time period inference request counts, per-node number of kubernetes (k8) pods running, per-node machine learning models running, free processor memory space, processor utilization, or combinations thereof").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the resource prediction and workload provisioning of Nasr-Azadani to include the resource scheduling of Poothia to determine which node and which hardware accelerator on which to schedule a ML model to run efficiently (see Poothia at paragraph 4).
As to dependent claim 8, Poothia further teaches determining which resources of the two or more resources are to be provisioned for operation of the machine learning model further comprises: for each respective resource of the two or more resources: determining whether the extent to which the operation of the machine learning model utilizes the respective resource based on the machine learning model and the resource constraints (Paragraph 12, "identify candidate hardware resources, from a plurality of hardware resources, capable of executing the machine learning model"); and based on the determined extent to which the machine learning model utilizes the respective resource, determining whether the multiple model containers are able to use the respective resource at the same time (Paragraph 58, "the identification may be resource-based, topology-based, or combinations thereof. In some examples, the identification may be resource-based, such as the presence of specific processors, whether the free memory available on the specific processors is greater than the machine learning model's memory requirement, and whether the specific processor's cores can accommodate running a new machine learning model without degrading the performance of models already running on the specific processor"); and determining which resources of the two or more resources are to be provisioned for operation of the machine learning model based on the one or more resource constraints, the indication of two or more resources, and the determinations of whether multiple model containers are able to use each resource of the two or more resources (Paragraph 12, "To select which hardware resource from the identified candidate hardware resources on which to schedule the machine learning model, AI inference service may, using a scoring algorithm, calculate (e.g., determine) a score for each identified candidate hardware resource").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the resource prediction and workload provisioning of Nasr-Azadani to include the resource scheduling of Poothia to determine which node and which hardware accelerator on which to schedule a ML model to run efficiently (see Poothia at paragraph 4).
As to dependent claim 9, Nasr-Azadani further teaches the method for provisioning machine learning model resources further comprises: causing operation of the machine learning model with the provisioned resources (Claim 1, "automatically schedule the user task according to the scheduling action." Claim 2, "the user task comprises a machine learning model").
As to independent claim 10, Nasr-Azadani teaches:
One or more storage devices collectively storing a machine learning model resource provisioning data structure, the data structure comprising:
information specifying a machine learning model (Claim 2, "the user task comprises a machine learning model");
information specifying one or more resource constraints (Paragraph 35, "specified target metrics or constraints (hardware usage, cost, time constraints, accuracy, and the like)");….
Nasr-Azadani does not appear to expressly teach information specifying two or more resources available for operation of the machine learning model, such that the information specifying the one or more resource constraints and the information specifying the two or more resources are usable to determine which resources of the two or more resources are to be provisioned for operation of the machine learning model.
Poothia teaches information specifying two or more resources available for operation of the machine learning model, such that the information specifying the one or more resource constraints and the information specifying the two or more resources are usable to determine which resources of the two or more resources are to be provisioned for operation of the machine learning model (Paragraph 12, "identify candidate hardware resources, from a plurality of hardware resources, capable of executing the machine learning model." Paragraph 12, "To select which hardware resource from the identified candidate hardware resources on which to schedule the machine learning model, AI inference service may, using a scoring algorithm, calculate (e.g., determine) a score for each identified candidate hardware resource").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the resource prediction and workload provisioning of Nasr-Azadani to include the resource scheduling of Poothia to determine which node and which hardware accelerator on which to schedule a ML model to run efficiently (see Poothia at paragraph 4).
As to independent claim 13, Nasr-Azadani teaches:
A system for provisioning machine learning model resources, the system comprising: a computing device configured to (Title and abstract):
receive an indication of a machine learning model (Claim 1, "receive input data, the input data comprising platform-independent task parameters and target metrics associated with a user task." Claim 2, "the user task comprises a machine learning model");
receive an indication of one or more resource constraints (Paragraph 35, "specified target metrics or constraints (hardware usage, cost, time constraints, accuracy, and the like)");….
Nasr-Azadani does not appear to expressly teach receive an indication of two or more resources available for operation of the machine learning model; and determine which resources of the two or more resources are to be provisioned for operation of the machine learning model based on the one or more resource constraints and the indication of two or more resources.
Poothia teaches receive an indication of two or more resources available for operation of the machine learning model (Paragraph 12, "identify candidate hardware resources, from a plurality of hardware resources, capable of executing the machine learning model"); and determine which resources of the two or more resources are to be provisioned for operation of the machine learning model based on the one or more resource constraints and the indication of two or more resources (Paragraph 12, "To select which hardware resource from the identified candidate hardware resources on which to schedule the machine learning model, AI inference service may, using a scoring algorithm, calculate (e.g., determine) a score for each identified candidate hardware resource").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the resource prediction and workload provisioning of Nasr-Azadani to include the resource scheduling of Poothia to determine which node and which hardware accelerator on which to schedule a ML model to run efficiently (see Poothia at paragraph 4).
As to dependent claim 14, Nasr-Azadani further teaches automatically cause the determined resources to be provisioned for operation of the machine learning model (Claim 1, "automatically generate a computing architecture and computing platform selection and computing resource prediction based on the input data and a hardware profile of the computer cluster using a computing resource prediction engine; automatically map the user task to one or more of the set of computing platforms and to a scheduling action among a set of scheduling actions using a trained intelligent computing resource assignment engine; automatically schedule the user task according to the scheduling action").
As to dependent claim 16, Nasr-Azadani further teaches receive an indication that at least a portion of the two or more resources available for operation of the machine learning model have changed (Paragraph 40, "A telemetry Application Programming Interface (API) 470 associated with the HPC cluster 450 may be further provided for monitoring the status of the HPC cluster 450 with respect to, for example, hardware operating and usage status of the HPC cluster 450, hardware/software license consumption status, network and storage status, and status of the user tasks and jobs, as shown in FIG. 4"); determine which new resources of the two or more resources are to be provisioned for operation of the machine learning model based on the one or more resources constraints and the two or more resources (Paragraph 40, "Such information may be fed back via the telemetry APIs to the intelligent job/task assignment engine 430 for dynamic and intelligent adjustment of resource allocation, user task re-routing, user task termination, and other actions"); and cause the new resources to be provisioned for operation of the machine learning model (Paragraph 40, "adjustment of resource allocation, user task re-routing, user task termination, and other actions").
As to dependent claim 18, Nasr-Azadani further teaches the indication that at least a portion of the two or more resources have changed includes one or more of: an indication that at least a portion of the one or more resource constraints have changed; an indication that software used to determine which resources are to be provisioned for operation of the machine learning model have changed; and an indication that at least one aspect of the two or more resources available for operation of the machine learning model have changed (Paragraph 40, "A telemetry Application Programming Interface (API) 470 associated with the HPC cluster 450 may be further provided for monitoring the status of the HPC cluster 450 with respect to, for example, hardware operating and usage status of the HPC cluster 450, hardware/software license consumption status, network and storage status, and status of the user tasks and jobs, as shown in FIG. 4." Paragraph 60, "The intelligent job/task assignment function may further consider unpredicted uncertainties 840 and may be based on the assignment policies 850. The uncertainties for example, may include unpredicted hardware failures and inaccurate resource predictions by the resource prediction twin." Paragraph 16, "such as early termination of running jobs which frees up previously occupied computing resources").
As to dependent claim 20, Nasr-Azadani further teaches for each respective node of the plurality of nodes: cause at least one model container of the plurality of model containers to be deployed to the respective node (Paragraph 31, "deployed user AI model").
Nasr-Azadani does not appear to expressly teach a plurality of nodes, each node including at least a portion of the determined resources; and the computing device is further caused to: generate a plurality of model containers based on the determined resources and the plurality of nodes, each model container including at least a portion of the machine learning model.
Poothia further teaches a plurality of nodes, each node including at least a portion of the determined resources; and the computing device is further caused to (Claim 5, "calculating based at least on a weighted sum of the execution priorities for each of the identified candidate hardware resources"): generate a plurality of model containers based on the determined resources and the plurality of nodes, each model container including at least a portion of the machine learning model (Claim 6, "the execution priorities comprise per-node per-time period inference request counts, per-node number of kubernetes (k8) pods running, per-node machine learning models running, free processor memory space, processor utilization, or combinations thereof").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the resource prediction and workload provisioning of Nasr-Azadani to include the resource scheduling of Poothia to determine which node and which hardware accelerator on which to schedule a ML model to run efficiently (see Poothia at paragraph 4).
As to dependent claim 21, Poothia further teaches for each respective resource of the two or more resources: determine whether the extent to which the operation of the machine learning model utilizes the respective resource based on the machine learning model and the resource constraints (Paragraph 12, "identify candidate hardware resources, from a plurality of hardware resources, capable of executing the machine learning model"); and based on the determined extent to which the machine learning model utilizes the respective resource, determine whether the multiple model containers are able to use the respective resource at the same time (Paragraph 58, "the identification may be resource-based, topology-based, or combinations thereof. In some examples, the identification may be resource-based, such as the presence of specific processors, whether the free memory available on the specific processors is greater than the machine learning model's memory requirement, and whether the specific processor's cores can accommodate running a new machine learning model without degrading the performance of models already running on the specific processor"); and determine which resources of the two or more resources are to be provisioned for operation of the machine learning model based on the one or more resource constraints, the indication of two or more resources, and the determinations of whether multiple model containers are able to use each resource of the two or more resources (Paragraph 12, "To select which hardware resource from the identified candidate hardware resources on which to schedule the machine learning model, AI inference service may, using a scoring algorithm, calculate (e.g., determine) a score for each identified candidate hardware resource").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the resource prediction and workload provisioning of Nasr-Azadani to include the resource scheduling of Poothia to determine which node and which hardware accelerator on which to schedule a ML model to run efficiently (see Poothia at paragraph 4).
Claims 2, 4, 6, 15, 17, and 19 are rejected under 35 U.S.C. § 103 as being unpatentable over Nasr-Azadani in view of Poothia and Song et al. (U.S. Pat. App. Pub. No. 2022/0083838, hereinafter Song).
As to dependent claim 2, the rejection of claim 1 is incorporated.
Nasr-Azadani does not appear to expressly teach the method for provisioning machine learning model resources further comprises: determining whether the machine learning model is optimized for operation with at least a portion of the determined resources; and based on the determining, optimizing the machine learning model for operation with the portion of the determined resources.
Song teaches the method for provisioning machine learning model resources further comprises: determining whether the machine learning model is optimized for operation with at least a portion of the determined resources; and based on the determining, optimizing the machine learning model for operation with the portion of the determined resources (Paragraph 6, "a neural network inference optimization, determining the predicted input data number to be a batch size of the subsequent interval, determining whether pipelining is to be performed in a target device based on a resource state of the target device, and applying, to the target device, an inference policy").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the resource prediction and workload provisioning of Nasr-Azadani to include the ML optimization of Song to enable the accelerator to perform inference with an optimized execution code (see Song at paragraph 4).
As to dependent claim 4, the rejection of claim 3 is incorporated.
Nasr-Azadani does not appear to expressly teach causing the new resources to be provisioned for operation of the machine learning model further comprises: optimizing the machine learning model for operation with the new resources.
Song teaches causing the new resources to be provisioned for operation of the machine learning model further comprises: optimizing the machine learning model for operation with the new resources (Paragraph 12, "performing compiling on a neural network model based on the inference policy, and loading a result of the compiling onto the target device").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the resource prediction and workload provisioning of Nasr-Azadani to include the ML optimization of Song to enable the accelerator to perform inference with an optimized execution code (see Song at paragraph 4).
As to dependent claim 6, the rejection of claim 1 is incorporated.
Poothia further teaches determining which resources of the two or more resources are to be provisioned for operation of the machine learning model based on the one or more resource constraints, the indication of two or more resources, and the determinations of whether inputs to the machine learning model can be batched (Paragraph 12, "To select which hardware resource from the identified candidate hardware resources on which to schedule the machine learning model, AI inference service may, using a scoring algorithm, calculate (e.g., determine) a score for each identified candidate hardware resource").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the resource prediction and workload provisioning of Nasr-Azadani to include the resource scheduling of Poothia to determine which node and which hardware accelerator on which to schedule a ML model to run efficiently (see Poothia at paragraph 4).
Nasr-Azadani does not appear to expressly teach determining which resources of the two or more resources are to be provisioned for operation of the machine learning model further comprises: for each respective resource of the two or more resources: determining whether inputs to the machine learning model can be batched while the respective resource is being used for operation of the machine learning model based on the machine learning model, the respective resource, and the resource constraints.
Song teaches determining which resources of the two or more resources are to be provisioned for operation of the machine learning model further comprises: for each respective resource of the two or more resources: determining whether inputs to the machine learning model can be batched while the respective resource is being used for operation of the machine learning model based on the machine learning model, the respective resource, and the resource constraints (Paragraph 6, "determining the predicted input data number to be a batch size of the subsequent interval").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the resource prediction and workload provisioning of Nasr-Azadani to include the ML optimization of Song to enable the accelerator to perform inference with an optimized execution code (see Song at paragraph 4).
As to dependent claim 15, the rejection of claim 14 is incorporated.
Nasr-Azadani does not appear to expressly teach determine whether the machine learning model is optimized for operation with at least a portion of the determined resources; and based on the determining, optimize the machine learning model for operation on the portion of the determined resources.
Song teaches determine whether the machine learning model is optimized for operation with at least a portion of the determined resources; and based on the determining, optimize the machine learning model for operation on the portion of the determined resources (Paragraph 6, "a neural network inference optimization, determining the predicted input data number to be a batch size of the subsequent interval, determining whether pipelining is to be performed in a target device based on a resource state of the target device, and applying, to the target device, an inference policy").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the resource prediction and workload provisioning of Nasr-Azadani to include the ML optimization of Song to enable the accelerator to perform inference with an optimized execution code (see Song at paragraph 4).
As to dependent claim 17, the rejection of claim 16 is incorporated.
Nasr-Azadani does not appear to expressly teach optimize the machine learning model for operation with the new resources.
Song teaches optimize the machine learning model for operation with the new resources (Paragraph 12, "performing compiling on a neural network model based on the inference policy, and loading a result of the compiling onto the target device").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the resource prediction and workload provisioning of Nasr-Azadani to include the ML optimization of Song to enable the accelerator to perform inference with an optimized execution code (see Song at paragraph 4).
As to dependent claim 19, the rejection of claim 14 is incorporated.
Poothia further teaches determine which resources of the two or more resources are to be provisioned for operation of the machine learning model based on the one or more resource constraints, the indication of two or more resources, and the determinations of whether inputs to the machine learning model can be batched (Paragraph 12, "To select which hardware resource from the identified candidate hardware resources on which to schedule the machine learning model, AI inference service may, using a scoring algorithm, calculate (e.g., determine) a score for each identified candidate hardware resource").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the resource prediction and workload provisioning of Nasr-Azadani to include the resource scheduling of Poothia to determine which node and which hardware accelerator on which to schedule a ML model to run efficiently (see Poothia at paragraph 4).
Nasr-Azadani does not appear to expressly teach for each respective resource of the two or more resources: determine whether inputs to the machine learning model can be batched while the respective resource is being used for operation of the machine learning model based on the machine learning model, the respective resource, and the resource constraints.
Song teaches for each respective resource of the two or more resources: determine whether inputs to the machine learning model can be batched while the respective resource is being used for operation of the machine learning model based on the machine learning model, the respective resource, and the resource constraints (Paragraph 6, "determining the predicted input data number to be a batch size of the subsequent interval").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the resource prediction and workload provisioning of Nasr-Azadani to include the ML optimization of Song to enable the accelerator to perform inference with an optimized execution code (see Song at paragraph 4).
Claim 11 is rejected under 35 U.S.C. § 103 as being unpatentable over Nasr-Azadani in view of Poothia and Feldman et al. (U.S. Pat. App. Pub. No. 2022/0237506, hereinafter Feldman).
As to dependent claim 11, the rejection of claim 10 is incorporated.
Nasr-Azadani does not appear to expressly teach information specifying a plurality of model containers, each model container including at least a portion of the machine learning model, such that the information specifying the plurality of model containers is usable to cause the determined resources to be provisioned for operation of the machine learning model.
Feldman teaches information specifying a plurality of model containers, each model container including at least a portion of the machine learning model (Paragraph 29, "Container information can be data about an isolated computer program execution environment that is enabled by a computer's operating system, and which executes the main functionality of a machine-learning model." Paragraph 31, "information about which serving containers are expected to host specific machine-learning models and which serving containers actually host the specified machine-learning models"), such that the information specifying the plurality of model containers is usable to cause the determined resources to be provisioned for operation of the machine learning model (Paragraph 31, "load, rebalance, delete, distribute, and replicate machine-learning models in the serving containers 128-152, based on the following information").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the resource prediction and workload provisioning of Nasr-Azadani to include the container and model information of Feldman to select containers for executing models (see Feldman at paragraph 21).
Claim 12 is rejected under 35 U.S.C. § 103 as being unpatentable over Nasr-Azadani in view of Poothia and Beaudoin et al. (U.S. Pat. App. Pub. No. 2020/0065708, hereinafter Beaudoin).
As to dependent claim 12, the rejection of claim 10 is incorporated.
Nasr-Azadani does not appear to expressly teach information specifying a plurality of instances of the model, each instance of the model being optimized for operation using a set of resources.
Beaudoin teaches information specifying a plurality of instances of the model, each instance of the model being optimized for operation using a set of resources (Paragraph 13, "adjusting said first trained model to operate on another of said multiple hardware platforms to result in at least one second trained model." Paragraph 40 et seq.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the resource prediction and workload provisioning of Nasr-Azadani to include the ML model hardware configuration based optimization of Beaudoin to assist in determining practical data processing configurations for implementations of machine learning models (see Beaudoin at paragraph 7).
Conclusion
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Casey R. Garner whose telephone number is 571-272-2467. The examiner can normally be reached Monday to Friday, 8am to 5pm, Eastern Time.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached on 571-270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR to authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated-interview-request-air-form.
/Casey R. Garner/Primary Examiner, Art Unit 2123
