DETAILED ACTION
Claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
The Examiner thanks the Applicant for amending Claims 1, 11, and 17 to overcome the objections to Claims 1 and 17 (Remarks p. 7). The objections have been withdrawn.
Applicant’s following arguments with respect to the 35 U.S.C. 103 rejections (Remarks pp. 7-10) have been fully considered, but are moot in view of the Examiner’s new ground of rejections based on added references to address applicant’s amendments.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4-6, 11-12, and 14-16 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang (US 20230334320 A1) in view of Min (US 20230086563 A1), Stat-Trek (“Probability Rules”), Kozhaya (US 20210058455 A1), and Hadad (US 20140068077 A1).
Regarding Claim 1, Zhang teaches a method, comprising:
defining a candidate set of one or more experiences for a workload including one or more inputs associated with features of the workload, the one or more experiences being to be analyzed (
Zhang discloses, “The NAS system 102 repeats the above-described process plural times until a prescribed condition is reached. For example, the NAS system 102 can repeat the process a predetermined number of times. Or the NAS system 102 can repeat the process until a prescribed number of candidate models have been identified that satisfy prescribed performance metrics. Once this decision is reached, a model-selecting component 124 can identify the subgroup of candidate models that satisfies a prescribed latency requirement, e.g., which offer latency performance below a prescribed latency threshold. The model-selecting component 124 can then select the candidate model within this subgroup that has the highest accuracy [Examiner’s notes: each model is comprised of several different layers or nodes that can have different levels of optimization. The claimed microservices are mapped to the layers or nodes.],” ¶ 0042, and “That is, assume that the attention component 210 receives input information in the form of a collection of input vectors, e.g., representing a series of respective text tokens,” ¶ 0044.
Zhang further teaches the NLP-based workload/task having features, stating “In some implementations, the technique includes: receiving a specified latency constraint; using neural architecture search to produce the chosen machine-trained model that satisfies the latency constraint, based on a collection of candidate machine-trained models; and applying the chosen machine-trained model in a computer-implemented application system to perform an application task,” ¶ 0002, “The base model 104 generally represents any machine-trained model having weights that have undergone at least some prior training. In some implementations, for example, a preliminary training system (not shown) can train the base model 104 to perform an application-agnostic natural language processing (NLP) task. For example, the preliminary training system can train the base model 104 to predict the identity of words that have been masked in a corpus of linguistic training examples,” ¶ 0031, “To repeat, the first relevance-processing component 910 is devoted to the task of processing target items for which encoding vectors currently exist,” ¶ 0085, “Referring to the first relevance-processing component 910, this component includes a first processing path 1002 for converting an input query into a query encoding vector 1004,” ¶ 0086, and “an embedding component 1010 breaks the input query into text tokens,” ¶ 0087.
Zhang further teaches the candidate models performing application tasks, stating, “In block 1108, the chosen machine-trained model is applied in a computer-implemented application system 902 to perform an application task [Examiner’s Note: application task is mapped to “workload”].” ¶ 0099. Zhang further teaches the candidate models comprising layers and nodes, stating “A layer-predicting component 310 maps the first hidden state information produced by the first encoding component 308 to layer mutation probability information, which indicates the suitability of each layer of the parent model for mutation. The layer-predicting component 310 then selects the single layer having the highest probability,” ¶ 0052.
The text describes the characterization or content of the application task, so the text has the features associated with the application task.
The claimed “workload” is mapped to the disclosed “application task”, which will be executed by a chosen machine-trained model comprising layers/nodes. Said application task is a natural language processing (NLP) task, as indicated by the similarities between paragraphs 2 and 31 regarding describing a task.
The claimed “one or more inputs associated with features of the workload” is mapped to the disclosed “input vectors” that are associated with the features, or characterization/content of the application task.
The claimed “one or more microservices” is mapped to the disclosed layers or nodes of the candidate model. This mapping is consistent with the specification, which states “a probability P of using each of the microservices M1, M2, M3, and M4 for executing the workload 120 given the computing resources associated with each of the microservices” (¶ 0041). Fig. 2 shows that M1-4 are nodes of a machine learning model.
The claimed “one or more experiences” is mapped to the disclosed association between application task, mapped to claimed “workload” and layers/nodes, mapped to claimed “microservices.” The application tasks are assigned to machine learning model’s layers and nodes to run.
The “one or more experiences” are defined when a model is selected to perform an application task, an association is built between the application task (workload) and nodes/layers (microservices) of the model. The “one or more experiences” are also defined when layers of a model are mutated, which means an old layer, and possibly some of its nodes, is no longer associated with the application task after the mutation; and a new mutated layer, and possibly its new node(s), becomes associated with the application task due to the mutation.
The current set of associations between application tasks and layer/nodes represents a candidate set of experiences. After associations have been removed or added until the set of associations can no longer be modified, this set of associations is a pruned set of experiences, as associations that are not optimal have been removed.);
wherein at a second ML model, a conditional probability of using each of the microservices of the one or more experiences to execute the workload given the one or more inputs of the workload, wherein the higher the conditional probability of one microservice is, the more likely that the one microservice executes the workload than other microservices (
Zhang discloses, “The mutating model is specifically trained to select a part of the parent model, and then to mutate the selected part… The technique repeats the above-identified operations to produce the final chosen machine-trained model, referred to herein as a neural architecture search (NAS) generated model,” ¶ 0003, “the mutating component 114 mutates the parent model chosen by the parent-selecting component 112, to produce a child model. In some implementations, the mutating component 114 is implemented as a machine-trained model (e.g., the “mutating model” 116 shown in FIG. 1),” ¶ 0051, and “An embedding component 306 can use a linear transform to transform the input vector into an embedding vector… A layer-predicting component 310 maps the first hidden state information produced by the first encoding component 308 to layer mutation probability information, which indicates the suitability of each layer of the parent model for mutation [Examiner’s notes: if a layer is mutated, it means that the original layer is replaced with an updated layer that is more optimized compared to the original.],” ¶ 0052. As Fig. 3 shows, the embedding, first encoding, and layer-predicting components are part of the mutating component.
The claimed “second ML model” is mapped to the disclosed “mutating component”, which could be implemented as a “machine-trained model”. The probability is generated at the mutating component.
The claimed “conditional probability” is mapped to the disclosed (1-“layer mutation probability”), which indicates the suitability of each layer (and/or nodes that make up each layer) of the parent model (which is chosen from a candidate model) for mutation. When the mutation probability is low, more likely the associated layer (microservice) will not be mutated and will execute the workload Therefore, when (1-“layer mutation probability”), mapped to conditional probability, is higher, more likely that the microservice/layer executes the workload, because it is less likely that the microservice/layer will be changed.
Here, the Examiner will introduce a secondary reference to teach: (1-“layer mutation probability”) = layer not-mutation probability = layer-will-be-unchanged-and-used probability.);
using a conditional probability to determine which of the one or more experiences will generate a low reward when analyzed (
Zhang discloses, “A reward-assessing component 118 determines a reward score for the child model identified by the mutating component 114.” ¶ 0039. The reward-assessing component can determine if and when the layers or nodes, that the child model comprises, generate a low reward when running a workload. Layers or nodes that have a higher mutation probability will be more likely to generate a low reward as they still have not been optimized further, while layers or nodes that have a lower mutation probability will be more likely to generate a higher reward due to already being optimized.);
removing first experiences that will generate the low reward from the candidate set to generate a pruned set (
Zhang discloses, “…the mutating component 114 selects a layer of the parent model. In a second stage, the mutating component 114 specifies how the selected layer is to be mutated,” ¶ 0038. Here, an individual layer/node is removed and replaced, as if it can be mutated, it is more likely to be unoptimized at first and thus would generate a low reward than an already-optimized layer/node.
Zhang also discloses, “For example, the population-updating component 122 can remove the oldest candidate model from the population, or the candidate model with the lowest reward score, etc.” ¶ 0041. The layers and nodes that make up the model, and generate the lowest reward score when running a workload, are removed.
The current set of associations between application tasks and layer/nodes represents a candidate set of experiences. After associations have been removed or added until the set of associations can no longer be modified, this set of associations is a pruned set of experiences, as associations that are not optimal have been removed.);
and analyzing second experiences that have not been removed from the candidate set to converge to find a goodreduce (
Zhang discloses, “…the mutating component 114 selects a layer of the parent model.” ¶ 0038. The unselected layers are less likely to be mutated due to being more optimized than the selected layer; thus, the unselected layers are more likely to be analyzed without modification.
Zhang also discloses, “Or the NAS system 102 can repeat the process until a prescribed number of candidate models have been identified that satisfy prescribed performance metrics. Once this decision is reached, a model-selecting component 124 can identify the subgroup of candidate models that satisfies a prescribed latency requirement, e.g., which offer latency performance below a prescribed latency threshold,” ¶ 0042.
Zhang discloses, “…the population-updating component 122 can remove the oldest candidate model from the population, or the candidate model with the lowest reward score, etc.,” ⁋ 0041.
The claimed “second experiences that have not been removed from the one or more experiences” is mapped to the retained associations between the layers and the workloads they are assigned to, after other associations have been selected and removed.
“converge to find a good experience” is mapped to the disclosed completion of the repeated process when a prescribed number of candidate models satisfy the prescribed performance metrics.
“reduce a system response time or a system latency” is mapped to the disclosed “prescribed latency requirement” being satisfied via the selected candidate models.),
.
Zhang does not teach:
the one or more experiences being to be analyzed at a first machine-learning (ML) model, after pruning;
generating a conditional probability at a second ML model;
using a conditional probability to determine which of the one or more experiences will generate a low reward when analyzed by the first ML model;
and analyzing at the first ML model only the pruned set including second experiences that have not been removed from the candidate set to converge to find a best experience which includes one or more microservices that should be used to execute the workload so as to minimize a system response time or a system latency.
Zhang implicitly teaches analyzing at the first ML model only the pruned set including second experiences that have not been removed from the candidate set, wherein the first experiences are never analyzed at the first ML model, but said teaching is not explicit.
However, Min teaches the one or more experiences being to be analyzed at a first machine-learning (ML) model, after pruning; using a conditional probability to determine which of the one or more experiences will generate a low reward when analyzed by the first ML model; and analyzing at the first ML model second experiences that have not been removed from the candidate set (
Min discloses, “That is, the reward is a reward score for an action (behavior) which the agent 10 determines based on any state when learning is performed through the reinforcement model,” ¶ 0010.
The claimed “first machine-learning (ML) model” is mapped to the disclosed “reinforcement model”.
After the combination of Zhang with Min, Min’s “reinforcement model” is used so that Zhang’s one or more experiences are to be analyzed by the “reinforcement model”, mapped to the claimed “first machine-learning (ML) model”, after pruning.
The Examiner’s primary reference already teaches using conditional probability to determine which of the one or more experiences will generate a low reward when analyzed. Those experiences will be pruned so that they do not have to be analyzed by the first ML model.).
Zhang and Min are both considered to be analogous to the claimed invention because they are in the same field of workload placement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang to incorporate the teachings of Min and provide the one or more experiences being to be analyzed at a first machine-learning (ML) model; using a conditional probability to determine which of the one or more experiences will generate a low reward when analyzed by the first ML model; and analyzing at the first ML model the pruned set including second experiences that have not been removed from the candidate set. Doing so would help automate the workload placement and improve efficiency of the overall process. After Zhang and Min are combined, Min’s “reinforcement model” for determining a reward score replaces Zhang’s “neural architecture search (NAS) system” (which comprises both the model-selecting component and the reward-assessing component) for determining a reward score.
Zhang in view of Min does not teach:
generating the conditional probability (1-“layer mutation probability”).
and analyzing at the first ML model only the pruned set including second experiences that have not been removed from the candidate set to converge to find a best experience which includes one or more microservices that should be used to execute the workload so as to minimize a system response time or a system latency;
wherein the first experiences are never analyzed at the first ML model.
Zhang in view of Min implicitly teaches analyzing at the first ML model only the pruned set including second experiences that have not been removed from the candidate set, wherein the first experiences are never analyzed at the first ML model, but said teaching is not explicit.
However, Stat-Trek teaches generating the conditional probability (1-“layer mutation probability”) (
PNG
media_image1.png
302
755
media_image1.png
Greyscale
Page 2.
After the combination of Zhang in view of Min and Stat-Trek, event A will not occur is mapped layer mutation probability, which layer not used for execution. Therefore, (1-“layer mutation probability”) = layer will be used for execution, because it will not be mutated.).
Zhang in view of Min, and Stat-Trek are both considered to be analogous to the claimed invention because they are in the same field of mathematics. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang in view of Min to incorporate the teachings of Stat-Trek. It would have been obvious to try (KSR): choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success. Both the “layer mutation probability” and (1-“layer mutation probability”) are known and predictable solutions based on Zhang in view of Min, and Stat-Trek. The use either has a reasonable expectation of success, e.g., by using minimum or maximum probability threshold to compare with “layer mutation probability” and (1-“layer mutation probability”).
Zhang in view of Min and Stat-Trek does not teach:
and analyzing at the first ML model only the pruned set including second experiences that have not been removed from the candidate set to converge to find a best experience which includes one or more microservices that should be used to execute the workload so as to minimize a system response time or a system latency;
wherein the first experiences are never analyzed at the first ML model.
Zhang in view of Min and Stat-Trek implicitly teaches analyzing at the first ML model only the pruned set including second experiences that have not been removed from the candidate set, wherein the first experiences are never analyzed at the first ML model, but said teaching is not explicit.
However, Kozhaya teaches to converge to find a best experience so as to minimize a system response time or a system latency (
Kozhaya discloses, “Using the objective function above to minimize microservice response time, illustrative embodiments are able to determine the optimal or best geographic location (i.e., data center) to deploy the microservice so that response time by the microservice is minimized,” ¶ 0051.).
Zhang in view of Min and Stat-Trek, and Kozhaya are both considered to be analogous to the claimed invention because they are in the same field of workload placement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang in view of Min and Stat-Trek to incorporate the teachings of Kozhaya and provide to converge to find a best experience which includes one or more microservices that should be used to execute the workload so as to minimize a system response time or a system latency. Doing so would help improve efficiency of the system by minimizing latency (Kozhaya discloses, “Thus, illustrative embodiments provide one or more technical solutions that overcome a technical problem with optimally deploying microservices to minimize response time within a network,” ¶ 0052.).
Zhang in view of Min, Stat-Trek, and Kozhaya does not explicitly teach:
and analyzing at the first ML model only the pruned set including second experiences that have not been removed from the candidate set;
wherein the first experiences are never analyzed at the first ML model.
Zhang in view of Min, Stat-Trek, and Kozhaya implicitly teaches analyzing at the first ML model only the pruned set including second experiences that have not been removed from the candidate set, wherein the first experiences are never analyzed at the first ML model, but said teaching is not explicit.
However, Hadad explicitly teaches analyzing at the first ML model only the pruned set including second experiences that have not been removed from the candidate set; wherein the first experiences are never analyzed at the first ML model (
Hadad discloses, “In accordance with one embodiment, the efficiency of the placement engine 120 may be increased by reducing the workload on the placement engine 120 in a manner such that instead of having to analyze all the variable elements noted above with respect to placement and allocation for a large plurality of N hosts in a cluster, a subset of the hosts (i.e., M hosts out of the total N hosts in the cluster) that are among the most suitable to service the request are selected. The selected M hosts are then presented to the placement engine 120 and the placement engine selects one or more of the M hosts to service the request,” ¶ 0017.
Here, a candidate set consists of the entire set of possible hosts that can service a request, or run a workload. The association of a host with a workload that the host will run is analogous to Zhang’s disclosed association between layers/nodes (microservices) and the application task to be run by the layers/nodes.
The pruned set consists of the subset of the entire set of possible hosts that is most suitable to service the request, or run a workload. This pruned set is analyzed by the placement engine.
This means that the hosts that are not part of the pruned set are omitted from being analyzed by the placement engine from the start, thus saving computational resources.
After the combination of Zhang in view of Min, Stat-Trek, and Kozhaya, with Hadad, the placement engine from Hadad is replaced with the first ML model from Zhang in view of Min, Stat-Trek, and Kozhaya, so that the first ML model is used to only analyze the pruned set of second experiences, and that the layer/nodes that are not part of Zhang’s pruned set are omitted from being analyzed by the first ML model from the start, thus saving computational resources.).
Zhang in view of Min, Stat-Trek, and Kozhaya, and Hadad are both considered to be analogous to the claimed invention because they are in the same field of workload placement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang in view of Min, Stat-Trek, and Kozhaya to incorporate the teachings of Hadad and provide analyzing at the first ML model only the pruned set including second experiences that have not been removed from the candidate set; wherein the first experiences are never analyzed at the first ML model. Doing so would help improve the efficiency of the training by eliminating candidates that do not meet satisfactory requirements for the process, thus saving computational resources (Hadad discloses, “In accordance with one embodiment, the efficiency of the placement engine 120 may be increased by reducing the workload on the placement engine 120 in a manner such that instead of having to analyze all the variable elements noted above with respect to placement and allocation for a large plurality of N hosts in a cluster, a subset of the hosts (i.e., M hosts out of the total N hosts in the cluster) that are among the most suitable to service the request are selected,” ¶ 0017.).
Claim 11 is a non-transitory storage medium claim corresponding to the method Claim 1 (Zhang discloses, “However, the specific term ‘computer-readable storage medium’ expressly excludes propagated signals per se in transit, while including all other forms of computer-readable media,” ¶ 0111.). Therefore, Claim 11 is rejected for the same reason set forth in the rejection of Claim 1.
Regarding Claim 2, Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad teaches the method of claim 1, wherein the first ML model is a reinforcement learning (RL) model (
Min discloses, “That is, the reward is a reward score for an action (behavior) which the agent 10 determines based on any state when learning is performed through the reinforcement model,” ¶ 0010.).
Zhang and Min are both considered to be analogous to the claimed invention because they are in the same field of workload placement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang to incorporate the teachings of Min and provide wherein the first ML model is a reinforcement learning (RL) model. Doing so would help ensure that the overall workload placement process can succeed even given vastly different starting conditions.
Claim 12 is a non-transitory storage medium claim corresponding to the method Claim 2 (Zhang discloses, “However, the specific term ‘computer-readable storage medium’ expressly excludes propagated signals per se in transit, while including all other forms of computer-readable media,” ¶ 0111.). Therefore, Claim 12 is rejected for the same reason set forth in the rejection of Claim 2.
Regarding Claim 4, Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad teaches the method of claim 1, wherein the first ML model finds the best experience (See Claim 1’s analysis related to “best experience”) by performing the following: generating expected rewards including a first expected reward and a second expected reward (
Zhang discloses, “A reward-assessing component 118 determines a reward score for the child model identified by the mutating component 114,” ⁋ 0039.
The claimed “first expected reward” and “second expected reward” are mapped to disclosed reward scores that are calculated for two different child models.);
performing a first action on the workload, by an agent associated with the workload, when the first expected reward is higher than the second expected reward (
Zhang discloses, “…the population-updating component 122 can remove the oldest candidate model from the population, or the candidate model with the lowest reward score, etc.,” ⁋ 0041.
The claimed “first action on the workload” is mapped to the disclosed retention of the first model, as the reward score of the first model is higher than that of the second model, so the first model is retained, while the second model is removed.
The claimed “agent associated with the workload” is mapped to the software component that runs the application task.);
and performing the second action on the workload when the second expected reward is higher than the first expected reward (
Zhang discloses, “…the population-updating component 122 can remove the oldest candidate model from the population, or the candidate model with the lowest reward score, etc.,” ⁋ 0041.
The claimed “second action on the workload” is mapped to the disclosed removal of the first model, as the reward score of the second model is higher than that of the first model, so the first model is removed, while the second model is retained.).
Claim 14 is a non-transitory storage medium claim corresponding to the method Claim 4 (Zhang discloses, “However, the specific term ‘computer-readable storage medium’ expressly excludes propagated signals per se in transit, while including all other forms of computer-readable media,” ¶ 0111.). Therefore, Claim 14 is rejected for the same reason set forth in the rejection of Claim 4.
Regarding Claim 5, Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad teaches the method of claim 4, wherein the first action is to keep the workload at a current microservice and wherein the second action is to migrate the workload to a different microservice (
Zhang discloses, “…the population-updating component 122 can remove the oldest candidate model from the population, or the candidate model with the lowest reward score, etc.,” ⁋ 0041.
The claimed “keep the workload at a current microservice” is mapped to the disclosed retention of the first model, which in this case has the higher reward score and thus will be assigned to execute the application task.
The claimed “migrate the workload to a different microservice” is mapped to the disclosed removal of the first model, which in this case has the lower reward score and thus will be replaced by the second model with the higher reward score instead, for executing the application task.).
Claim 15 is a non-transitory storage medium claim corresponding to the method Claim 5 (Zhang discloses, “However, the specific term ‘computer-readable storage medium’ expressly excludes propagated signals per se in transit, while including all other forms of computer-readable media,” ¶ 0111.). Therefore, Claim 15 is rejected for the same reason set forth in the rejection of Claim 5.
Regarding Claim 6, Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad teaches the method of claim 4, wherein the first ML model comprises a neural network configured to map to expected rewards (
Zhang discloses, “The NAS system ultimately selects a neural network architecture that best satisfies specified performance objectives,” ⁋ 0001, and “The NAS system 102 performs RL [(reinforcement learning)] operations to the extent that it assigns reward scores to the models it mutates,” ⁋ 0036).
Zhang and Min are both considered to be analogous to the claimed invention because they are in the same field of workload placement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang to incorporate the teachings of Min and provide wherein a reinforcement learning (RL) model comprises a neural network, rather than the NAS system comprising a neural network. Doing so would help ensure that the overall reward mapping process can succeed even given vastly different starting conditions.
Claim 16 is a non-transitory storage medium claim corresponding to the method Claim 6 (Zhang discloses, “However, the specific term ‘computer-readable storage medium’ expressly excludes propagated signals per se in transit, while including all other forms of computer-readable media,” ¶ 0111.). Therefore, Claim 16 is rejected for the same reason set forth in the rejection of Claim 6.
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang (US 20230334320 A1) in view of Min (US 20230086563 A1), Stat-Trek (“Probability Rules”), Kozhaya (US 20210058455 A1), Hadad (US 20140068077 A1), and Durduran (US 20200090819 A1).
Regarding Claim 3, Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad teaches the method of claim 1. Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad does not teach wherein the second ML model is a Restricted Boltzmann Machine (RBM) model.
However, Durduran teaches wherein the second ML model is a Restricted Boltzmann Machine (RBM) model. (
Durduran discloses, “Restricted Boltzmann machines to develop a generative model that maximizes the probability of constructing data from the input layer by sampling a lower dimensional hidden layer.” ⁋ 0110.).
Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad, and Durduran are both considered to be analogous to the claimed invention because they are in the same field of machine learning model utilization. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad to incorporate the teachings of Durduran and provide wherein the second ML model is a Restricted Boltzmann Machine (RBM) model. Doing so would help increase the reliability of generating a low reward or high reward for each experience, in order to filter out the low-reward experiences more easily. (Durduran discloses, “Restricted Boltzmann machines to develop a generative model that maximizes the probability of constructing data from the input layer by sampling a lower dimensional hidden layer.” ⁋ 0110.)
Claim 13 is a non-transitory storage medium claim corresponding to the method Claim 3 (Zhang discloses, “However, the specific term ‘computer-readable storage medium’ expressly excludes propagated signals per se in transit, while including all other forms of computer-readable media,” ¶ 0111.). Therefore, Claim 13 is rejected for the same reason set forth in the rejection of Claim 3.
Claims 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang (US 20230334320 A1) in view of Min (US 20230086563 A1), Stat-Trek (“Probability Rules”), Kozhaya (US 20210058455 A1), Hadad (US 20140068077 A1), and Tortosa (US 20210092071 A1).
Regarding Claim 7, Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad teaches the method of claim 1. Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad does not teach wherein the best experience includes microservices placed at a geographical location that reduces the system response time or the system latency when executing the workload.
However, Tortosa teaches wherein the best experience includes microservices placed at a geographical location that reduces the system response time or the system latency when executing the workload. (Tortosa discloses, “Accordingly, in response to determining to relocate processing resources, embodiments of the present invention can relocate processing resources from an initial computing system to a computing system that is in a geographic location that reduces processing latency,” ⁋ 0012).
Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad, and Tortosa are both considered to be analogous to the claimed invention because they are in the same field of workload placement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad, to incorporate the teachings of Tortosa and provide wherein the best experience includes microservices placed at a geographical location that reduces the system response time or the system latency when executing the workload. Doing so would help increase the efficiency of the workload execution in a distributed neural network.
Claim 17 is a non-transitory storage medium claim corresponding to the method Claim 7 (Zhang discloses, “However, the specific term ‘computer-readable storage medium’ expressly excludes propagated signals per se in transit, while including all other forms of computer-readable media,” ¶ 0111.). Therefore, Claim 17 is rejected for the same reason set forth in the rejection of Claim 7.
Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang (US 20230334320 A1) in view of Min (US 20230086563 A1), Stat-Trek (“Probability Rules”), Kozhaya (US 20210058455 A1), Hadad (US 20140068077 A1), and Xiong (US 20180260746 A1).
Regarding Claim 8, Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad teaches the method of claim 1. Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad does not teach wherein generating the conditional probability comprises:
providing the one or more inputs into to the second ML model, the one or more inputs comprising one or more features of the workload;
determining the computing resources associated with each of the one or more microservices;
and using the inputs to determine the conditional probability for each of the one or more microservices given the computing resources associated with each microservice.
However, Xiong teaches wherein generating the conditional probability comprises:
providing the one or more inputs into to the second ML model, the one or more inputs comprising one or more features of the workload (
Xiong discloses, “a bivariate supervised learning model for determining the probability of a resource performing the job, where a job vector and a resource vector may be taken as features” ⁋ 0053);
determining the computing resources associated with each of the one or more microservices (
Xiong discloses, “where a job vector and a resource vector may be taken as features” ⁋ 0053);
and using the inputs to determine the conditional probability for each of the one or more microservices given the computing resources associated with each microservice (
Xiong discloses, “a bivariate supervised learning model for determining the probability of a resource performing the job, where a job vector and a resource vector may be taken as features to provide a probability of a resource performing a job.” ⁋ 0053).
Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad, and Xiong are both considered to be analogous to the claimed invention because they are in the same field of machine learning model utilization. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad, to incorporate the teachings of Xiong and provide wherein generating the conditional probability comprises: providing the one or more inputs into to the second ML model, the one or more inputs comprising one or more features of the workload; determining the computing resources associated with each of the one or more microservices; and using the inputs to determine the conditional probability for each of the one or more microservices given the computing resources associated with each microservice. Doing so would help determine which microservice is best suited for the workload.
Claim 18 is a non-transitory storage medium claim corresponding to the method Claim 8 (Zhang discloses, “However, the specific term ‘computer-readable storage medium’ expressly excludes propagated signals per se in transit, while including all other forms of computer-readable media,” ¶ 0111.). Therefore, Claim 18 is rejected for the same reason set forth in the rejection of Claim 8.
Claims 9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang (US 20230334320 A1) in view of Min (US 20230086563 A1), Stat-Trek (“Probability Rules”), Kozhaya (US 20210058455 A1), Hadad (US 20140068077 A1), Xiong (US 20180260746 A1), and Parizi (US 20220180275 A1).
Regarding Claim 9, Zhang in view of Min, Stat-Trek, Kozhaya, Hadad, and Xiong teaches the method of claim 8. Zhang in view of Min, Stat-Trek, Kozhaya, Hadad, and Xiong does not teach wherein the one or more features include a data type of the workload, computing resources needed to execute the workload, a geographical location of the workload, and a usage or execution pattern of the workload.
However, Parizi teaches wherein the one or more features include a data type of the workload, computing resources needed to execute the workload, a geographical location of the workload, and a usage or execution pattern of the workload. (Parizi discloses, “Thus, data center data store 110 may include… geographic location data, duration of time it takes to receive server clusters from server warehouses (e.g., from time of order), and cost to install and execute workloads on server clusters. Data center data store 110 may additionally include current and past customer workloads (e.g., number of virtual cores utilized by each customer, types of workloads handled by each server cluster or data center)…” ⁋ 0037.
The claimed “data type of the workload” is mapped to the disclosed “types of workloads”.
The claimed “computing resources needed to execute the workload” is mapped to the disclosed “cost to install and execute workloads on server clusters”.
The claimed “geographical location of the workload” is mapped to the disclosed “geographic location data”.
The claimed “usage or execution pattern of the workload” is mapped to the disclosed “current and past customer workloads”.
).
Zhang in view of Min, Stat-Trek, Kozhaya, Hadad, and Xiong, and Parizi are both considered to be analogous to the claimed invention because they are in the same field of workload placement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang in view of Min, Stat-Trek, Kozhaya, Hadad, and Xiong to incorporate the teachings of Parizi and provide wherein the one or more features include a data type of the workload, computing resources needed to execute the workload, a geographical location of the workload, and a usage or execution pattern of the workload. Doing so would allow for a more informed and more customized data service for the user.
Claim 19 is a non-transitory storage medium claim corresponding to the method Claim 9 (Zhang discloses, “However, the specific term ‘computer-readable storage medium’ expressly excludes propagated signals per se in transit, while including all other forms of computer-readable media,” ¶ 0111.). Therefore, Claim 19 is rejected for the same reason set forth in the rejection of Claim 9.
Claims 10 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang (US 20230334320 A1) in view of Min (US 20230086563 A1), Stat-Trek (“Probability Rules”), Kozhaya (US 20210058455 A1), Hadad (US 20140068077 A1), and Illikkal (US 20240015080 A1).
Regarding Claim 10, Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad teaches the method of claim 1. Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad does not teach wherein the computing resources include virtual machines, physical machines, GPUs, and CPUs configured to execute the workload.
However, Illikkal teaches wherein the computing resources include virtual machines, physical machines, GPUs, and CPUs configured to execute the workload. (
Illikkal discloses, “Further, the data center 100 may include pooled memory (not shown), accessible by the compute nodes 102, that supports data-intense applications by minimizing access latency and enabling in-memory processing, and/or pooled hardware accelerators (not shown), such as graphic processing units (GPUs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), or other special-purpose processors for performing certain routine, but computationally expensive computing tasks,” ⁋ 0035, and “Some software architectures utilize virtual machines. In the example of FIG. 18, this is illustrated by virtual machine 1848. A virtual machine creates a software environment where applications/modules can execute as if they were executing on a hardware computing device. A virtual machine is hosted by a host operating system (operating system 1814),” ⁋ 0330.)
Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad, and Illikkal are both considered to be analogous to the claimed invention because they are in the same field of machine learning model utilization. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang in view of Min, Stat-Trek, Kozhaya, and Hadad to incorporate the teachings of Illikkal and provide wherein the computing resources include virtual machines, physical machines, GPUs, and CPUs configured to execute the workload. Doing so would help ensure that the workload can be executed by utilizing a wide range of available resources.
Claim 20 is a non-transitory storage medium claim corresponding to the method Claim 10 (Zhang discloses, “However, the specific term ‘computer-readable storage medium’ expressly excludes propagated signals per se in transit, while including all other forms of computer-readable media,” ¶ 0111.). Therefore, Claim 20 is rejected for the same reason set forth in the rejection of Claim 10.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Castellanos et al. (US 7467145 B1): System and Method for Analyzing Processes
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW SUN whose telephone number is (571)272-6735. The examiner can normally be reached Monday-Friday 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Aimee Li can be reached at (571) 272-4169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW NMN SUN/Examiner, Art Unit 2195
/Aimee Li/Supervisory Patent Examiner, Art Unit 2195
Prosecution Insights