DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
(Submitted on 3/4/2026)
In regard to 101 Rejections
Examiner’s Response:
The applicant’s argument on Page 11 was considered. As a result of the amended claims indicating the
weight and biases as the parameters, the entire context of the claim limitations were considered as a
whole, the examiner submits that the claim limitations only recites the training using generic
computer functions.
In CONCLUSION , the examiner WITHDRAWS the 101 rejections on claims 1 and 18 and on all
dependent claims 3-4, 6, 8-12, 14-17 and 19-21.
In regard to 103 Rejections
The applicant further argues on Page 14 that virtual machine as a simulation environment does not
relate to the features as no actual algorithm execution is demonstrated.
Examiner’ Response
The examine respectfully disagree with the applicant arguments. Perhaps known to a POSITA that A VM
can indeed serve as a simulator for neural network performance testing, and with the hypervisor’s
resource monitoring capabilities plus OS-level profiling and can track hardware utilization per
execution and compute meaningful performance scores. The reference “Agra” teaches [Abstract]”
respective score is calculated by invoking the meta-model based on at least one of: a respective subset
of meta-feature values, and/or hyperparameter values of a respective subset of hyperparameters of the
algorithm”. Perhaps it is known to the POSITA that a high performance score with a given
hyperparameter configuration can be a proxy for good feature representation if the performance is
stable, high, and aligned with strong meta-feature indicators. Further, the examiner submits the
reference “Agra” teaches [0084] “ meta-model 151 was trained by observing the performance of algorithm 121 configured with fewer layers for monochromatic photography and meta-model 153 was trained by observing the performance of algorithm 121 configured with more layers for full-color photography” and further teaches [0096]” Scores 161-163 share a performance measurement scale. For example, a score may predictively measure how proficient (accuracy such as error rate) would a particular configuration of a particular algorithm become after training for a fixed duration with a particular training dataset” and further teaches [0162] “ in an embodiment. Computer 600 improves ensemble meta-learning with optimizations such as boosting and bagging”.
Thus, the examiner submits that overall, a comparative performance score that evaluates a meta-model with fewer layers and a subset of hyperparameters can indeed relate to the quality of the feature representation. Thus, the examiner further strongly affirm that model’s performance score is directly influenced by the quality and relevance of its feature representation where feature representation refers to how the raw data is transformed, encoded, and structured before being fed into the model. A well-chosen, meaningful feature set can improve performance.
The applicant argues on Page 14 that none of the references teaches the monitoring hardware resource
utilization of each neural network when executed in the respective simulation and argues that the
references do not teach selecting a group of neural networks.
Examiner’ Response:
The examiner respectfully disagree with the applicant. Perhaps known to a POSITA, selecting the best algorithm(s) based on a combined score from multiple meta-models is conceptually the same as selecting the best group of neural networks based on a combined score for each group. Using semantic scoring instead of raw accuracy or majority voting can significantly improve selection quality, especially in ambiguous or open-ended tasks.
Applicant’s arguments with respect to claims 1 and 18 amendments have been considered but are moot
because the new ground of rejection does not rely on any reference applied in the prior rejection of
record for any teaching or matter specifically challenged in the argument. The examiner uses a new reference “Hermann” to teach the amendments of claims 1 and 18 as related to adjusting shared parameters.
In CONCLUSION, the examiner rejects claims 1 and 18, and claims 3-4, 6, 8-12, 14-17 and 19-21 under
103 and MOVES the application to FINAL REJECTION.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-4, 8-11, 14, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over
in view of Sandeep Agrawal et. al. (hereinafter Agra) US 2019/0095756 A1,
in view of Hyung Taek RIM et. al. (hereinafter RIM) US 2019/0221313 A1,
in view of Moritz Karl Hermann et. al. (hereinafter Hermann) US 2021/0110115 A1.
In regards to Claim 1: (Currently Amended)
Agra discloses:
- A computer-implemented method of providing a group of neural networks for processing data in a plurality of hardware environments performed by a first processing system
[0183]:
In 0182] : For example, FIG. 7 is a block diagram that illustrates a computer system 700 upon which an embodiment of the invention may be implemented.
According to one embodiment, the techniques described herein are implemented by one or more special-purpose computing devices
[0007]:
FIG. 3 is a block diagram that depicts an example computer that trains an algorithm-specific machine learning ensemble for contextually accurate performance prediction, in an embodiment;
[0113]:
Computer 300 contains configurable trainable algorithms 321-323. Each algorithm is associated with its own machine learning ensemble, such as 390.
[Abstract]:
the algorithm(s) are selected based on the respective scores. Based on the inference dataset, the selected algorithm(s) may be invoked to obtain a result. In an embodiment, the trained regressors are distinctly configured artificial neural networks.
[0129]:
during inferencing after ensemble training, computer 300 may invoke multiple ensembles to detect how well each algorithm would perform for a same unfamiliar inference dataset
- comprising a plurality of parameters and wherein one or more of the parameters of each sub-neural network are shared by the sub-neural network and the main neural network;
such that the parameters of each sub-neural network comprise a respective subset of the plurality of parameters of the main neural network
[0181]:
meta-models 651-656 themselves have hyperparameters (not shown) such as count of layers if an ensemble is composed of neural networks or other hyperparameters if composed of another machine learning algorithm.
[0117]:
Although not shown, ensemble 390 may itself be a composite neural network that contains the neural networks of meta-models 351-352.
(BRI: Perhaps known to the POSITA that by embedding multiple neural networks (meta-models) into a composite network with a final layer for integration, you can create a powerful, flexible, and potentially more accurate ensemble model in which the each meta-model is a separate neural network and the composite meta model is a main neural network)
[0133] :
For example with tuples 331-333, only 332 is suitable for training both meta-models 351-352 as indicated by the dotted arrows flowing into meta-models 351-352. Multiple meta-models that accept a same shared tuple, such as 332, may take different subsets of (hyperparameter and meta-feature) values that are associated with the shared tuple.
[0074] :
In operation, computer 100 obtains inference dataset 110 and should use meta-models, such as 151-153, to select a more or less optimal subset of algorithms 121-123 to eventually be tuned with inference dataset 110. When predicting performance of an algorithm, a meta-model should consider features of the algorithm and features of inference dataset 110.
[0178] :
Boosting may also assign weights to meta-models 651-656 of an ensemble based on their accuracy. Thus, a more reliable meta-model may have more influence over an ensemble score than a less reliable meta-model.
[0179];
Meta-model weights can be adjusted based on observed accuracy at various times during training. In an embodiment, training ceases when most or all meta-model weights cease to change by at least a threshold amount.
[0108] :
After step 204 is sufficiently repeated, all meta-models of all algorithms 121-123 have scores. Based on those scores, at least one promising algorithm is selected for training. For example, computer 100 selects algorithm 122 that has the highest scoring meta-model of all algorithms or the highest mean, median, or modal score of all algorithms.
[0064]:
For each trainable algorithm and each regression meta-model that is respectively associated with the algorithm, a respective score is calculated by invoking the meta-model based on at least one of: a) a respective subset of meta-feature values
0093] :
a subset of meta-feature values as stimulus inputs, shown as dashed arrows entering meta-models 151-153. Training of meta-models is discussed later herein.
[0104] :
most application datasets consist of data units such as pixels, photographs, or tuples (e.g. database table rows). To the extent that a machine learning algorithm may have some configurations that adequately learn with little training data and other configurations that more accurately learn with much training data, one useful meta-feature may be the size of the dataset, such as a count of rows, pixels, or photographs.
[BRI: meta model weights are indeed parameters of a neural network, but they are learned or optimized in a special way to enable faster adaptation and better generalization across tasks . A subset of meta-feature values as stimulus input for the meta-model corresponds to a subset of the parameters of the main neural network, because the meta-features are derived from those parameters)
- computing a performance score for each neural network in the group of neural networks using the adjusted parameters by:
[0096]:
Scores 161-163 share a performance measurement scale. For example, a score may predictively measure how proficient (accuracy such as error rate) would a particular configuration of a particular algorithm become after training for a fixed duration with a particular training dataset, for which inference dataset 110 is representative (e.g. small sample) of the training dataset.
(BRI: accuracy is a performance score on which basis the selection is performed)
- inputting test data to each neural network, wherein each neural network is implemented in a respective simulation of respective hardware environment;[[,]] executing each neural network in the respective simulations of the respective hardware environments to process the test data;
[0093]:
For example, meta-models 151-153 may each be an already trained neural network that takes a subset of hyperparameter values and a subset of meta-feature values as stimulus inputs, shown as dashed arrows entering meta-models 151-153. Training of meta-models is discussed later herein.
- executing each neural network in the respective simulations of the respective hardware environments to process the test data;
[0091]:
stimulating already-trained meta-models with respective subsets of hyperparameter values and meta-feature values of a new (unfamiliar) inference dataset such as 110, computer 100 may detect how suitable are various hyperparameter configurations of various algorithms 121-123.
[0199] :
OS 810 can execute directly on the bare hardware 820 (e.g., processor(s) 704) of computer system 700. Alternatively, a hypervisor or virtual machine monitor (VMM) 830 may be interposed between the bare hardware 820 and the OS 810.
- and monitoring hardware resource utilization of each neural network when executed in the respective simulations of the respective hardware environments to compute the performance scores
In[ 0201]:
In some instances, the VMM 830 may allow a guest operating system to run as if it is running on the bare hardware 820 of computer system 800 directly. In these instances, the same version of the guest operating system configured to execute on the bare hardware 820 directly may also execute on VMM 830 without modification or reconfiguration. In other words, VMM 830 may provide full hardware and CPU virtualization to a guest operating system in some instances.
[0202]:
In other instances, a guest operating system may be specially designed or configured to execute on VMM 830 for efficiency. In these instances, the guest operating system is “aware” that it executes on a virtual machine monitor. In other words, VMM 830 may provide para-virtualization to a guest operating system in some instances.
repeating the identifying , the inputting, the adjusting, the computing and the generating, for two or more iterations wherein the single objective function is computed at each iteration that represents both the difference between output data generated at the output of each neural network and the expected output data and additionally performance scores of each neural network in the group of neural networks using the adjusted paramet;;
[0106]:
Step 204 is repeated for each trainable algorithm that is available to computer 100. For each meta-model associated with a same algorithm, step 204 calculates a respective score by invoking the meta-model based on at least one of: a) a respective subset of meta-feature values, or b) hyperparameter values of a respective subset of hyperparameters of algorithm.
[0107]:
For example, already-trained distinct meta-models 151-153 may be individually stimulated with a respective subset of meta-feature values 171-174 and a respective subset of hyperparameter values 141-146 as inference inputs. For example, meta-model 151 calculates score 161 based on meta-feature values 171-172 and hyperparameter values 142-143.
selecting from the plurality of groups of neural network generated by the repeating , a group of networks for processing data in the plurality of hardware environments based on the value of the combined score for each group of neural networks
[0098]:
Regardless of score semantics, each meta-model of each algorithm emits a score. Computer 100 may select the best one or few algorithms (perhaps also best hyperparameter values), such as 122 as shown, as ranked based on sorted scores.
- and deploying the selected group of neural networks to a second processing system processing data in the plurality of hardware environments, the selected group of neural networks
[0206]:
Depending on the particular implementation, the precise definition of components or features provided by or within each cloud service layer can vary, but common examples include: Software as a Service (SaaS), in which consumers use software applications that are running upon a cloud infrastructure, while a SaaS provider manages or controls the underlying cloud infrastructure and applications. Platform as a Service (PaaS), in which consumers can use software programming languages and development tools supported by a PaaS provider to develop, deploy, and otherwise control their own applications, while the PaaS provider manages or controls other aspects of the cloud environment (i.e., everything below the run-time execution environment). Infrastructure as a Service (IaaS), in which consumers can deploy and run arbitrary software applications, and/or provision processing, storage, networks, and other fundamental computing resources, while an IaaS provider manages or controls the underlying physical cloud infrastructure
Agra does not explicitly disclose:
- the method comprising: Identifying a group of neural networks including a main neural network and one or more sub-neural networks, each neural network in the group of neural networks comprising a plurality of parameters , the parameters comprising a plurality of weights and biases,
- inputting the same training data into each neural network in the group of neural networks;[[,]]
- adjusting the parameters of each neural network of the group of neural networks based on a backpropagation during a training process that uses a single objective function that represents differences between output data generated at an output of each of the group of networks, and expected output data
generating a combined score for the group of neural networks by combining the performance score of each neural network in the group of neural networks, with a value of a loss function computed for each neural network in the group of neural networks using the adjusted parameters;
including metadata representing a target hardware environment and/or hardware requirement, of each neural network of the group of neural networks.
However, RIM discloses in:
- the method comprising: Identifying a group of neural networks including a main neural network and one or more sub-neural networks, each neural network in the group of neural networks comprising a plurality of parameters, the parameters comprising a plurality of weights and biases,
[0189]:
an ensemble of the plurality of sub-neural network models may be determined as a final neural network model, and parameter values related to each of the plurality of sub-neural network models may be obtained as training results.
[0189]:
an ensemble of the plurality of sub-neural network models may be determined as a final neural network model, and parameter values related to each of the plurality of sub-neural network models may be obtained as training results.
[0185]: an embodiment of the present invention may be implemented by combining a plurality of sub-neural network models. In other words, training of a neural network model may be performed using an ensemble technique in which a plurality of sub-neural network models are combined.
[0175]:
A neural network model may be validated using a validation data set. Validation of a neural network model may be performed by obtaining a result value related to a validation data set from a neural network model which has been trained and comparing the result value with a label of the validation data set. The validation may be performed by measuring accuracy of the result value. Parameters of a neural network model (for example, weights and/or bias) or hyperparameters (for example, learning rate) of the neural network model may be adjusted according to a validation result.
- inputting the same training data into each neural network in the group of neural networks;[[,]]
[0189]:
According to an embodiment of the present invention, a plurality of sub-neural network models may obtain the same training data set and individually generate output values. In this case, an ensemble of the plurality of sub-neural network models may be determined as a final neural network model, and parameter values related to each of the plurality of sub-neural network models may be obtained as training results. An output value of the final neural network model may be set to an average value of the output values by the sub-neural network models. Alternatively, in consideration of accuracy obtained as a result of validating each of the sub-neural network models, the output value of the final neural network model may be set to a weighted average value of the output values of the sub-neural network models.
- adjusting the parameters of each neural network of the group of neural networks based on a backpropagation during a training process that uses a single objective function that represents differences between output data generated at an output of each of the group of networks, and expected output data,
[0189]:
a plurality of sub-neural network models may obtain the same training data set and individually generate output values. In this case, an ensemble of the plurality of sub-neural network models may be determined as a final neural network model, and parameter values related to each of the plurality of sub-neural network models may be obtained as training results. An output value of the final neural network model may be set to an average value of the output values by the sub-neural network models. Alternatively, in consideration of accuracy obtained as a result of validating each of the sub-neural network models, the output value of the final neural network model may be set to a weighted average value of the output values of the sub-neural network models.
[0173]:
Training of a neural network model may be performed by obtaining a result value using a neural network model to which arbitrary weights are assigned on the basis of training image data, comparing the obtained result value with a label value of the training data, and performing backpropagation according to an error therebetween to optimize the weights,
[0189]:
According to an embodiment of the present invention, a plurality of sub-neural network models may obtain the same training data set and individually generate output values. In this case, an ensemble of the plurality of sub-neural network models may be determined as a final neural network model, and parameter values related to each of the plurality of sub-neural network models may be obtained as training results. An output value of the final neural network model may be set to an average value of the output values by the sub-neural network models. Alternatively, in consideration of accuracy obtained as a result of validating each of the sub-neural network models, the output value of the final neural network model may be set to a weighted average value of the output values of the sub-neural network models.
generating a combined score for the group of neural networks by combining the performance score of each neural network in the group of neural networks, with a value of a loss function computed for each neural network in the group of neural networks using the adjusted parameters;
[0191]:
any one of the plurality of sub-neural network models may be selected on the basis of the accuracy and determined as the final neural network model. A structure of the determined sub-neural network model and parameter values of the determined sub-neural network model obtained as a result of training may be stored.
[0173]:
Training of a neural network model may be performed by obtaining a result value using a neural network model to which arbitrary weights are assigned on the basis of training image data, comparing the obtained result value with a label value of the training data, and performing backpropagation according to an error therebetween to optimize the weights,
[0189]:
According to an embodiment of the present invention, a plurality of sub-neural network models may obtain the same training data set and individually generate output values. In this case, an ensemble of the plurality of sub-neural network models may be determined as a final neural network model, and parameter values related to each of the plurality of sub-neural network models may be obtained as training results. An output value of the final neural network model may be set to an average value of the output values by the sub-neural network models. Alternatively, in consideration of accuracy obtained as a result of validating each of the sub-neural network models, the output value of the final neural network model may be set to a weighted average value of the output values of the sub-neural network models.
including metadata representing a target hardware environment and/or hardware requirement, of each neural network of the group of neural networks.
[0184]:
in a process of training a single diagnosis assistance neural network model, a plurality of sub-models may be simultaneously trained. The plurality of sub-models may have different layer structures.
[0185]:
training of a neural network model may be performed using an ensemble technique in which a plurality of sub-neural network models are combined.
[0162]:
A data set may be obtained from a queue. The data set may be obtained in batches from the queue. For example, when sixty data sets are designated as the size of a batch, sixty data sets may be extracted at a time from the queue. The size of a batch may be limited by the capacity of a RAM of a GPU.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, and RIM.
Agrawal teaches ensemble training on a same training set and deployment on the target hardware.
RIM teaches metadata of hardware.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra and RIM to improve the prediction result can be improved ( RIM [0186]).
Agra and RIM do not explicitly disclose:
- wherein the adjusting comprises simultaneously adjusting the shared parameters for both the main neural network and one or more sub-neural networks
However, Hermann discloses in:
- wherein the adjusting comprises simultaneously adjusting the shared parameters for both the main neural network and one or more sub-neural networks
[0026]:
In some implementations, the temporal autoencoder neural network shares one or more parameters with the action selection neural network, and determining the auxiliary update further includes determining an update to the current values of the shared parameters.
[BRI: in deep reinforcement learning (DRL) systems, temporal autoencoders (often used for feature extraction, denoising, or latent representation learning) and action selection networks (e.g., policy networks) can share parameters and is a common practice in modular or multi-task neural network designs)
[0007]:
the subsystem combines the current observation embedding and the current text embedding to generate a current combined embedding. The subsystem selects, using the current combined embedding, an action to be performed by the agent in response to the current observation.
[0016]:
In some implementations, the current observation embedding is a feature matrix of the current observation, and wherein the current text embedding is a feature vector of the current text string.
[0016] :
In some implementations, the current observation embedding is a feature matrix of the current observation, and wherein the current text embedding is a feature vector of the current text string.
[0040] :
In some implementations, the computing system includes a combining module between the first and second neural network modules and the policy defining neural network module. The combining module has inputs coupled to the first and second neural network modules to combine the environment feature data and the embedded representation of the task input data, and is configured to output combined representation data for the policy defining neural network
[0078]:
At each time step, the system 100 combines the current observation embedding 114 and the current text embedding 118 to determine a current combined embedding 120, and uses the current combined embedding 120 to select an action 102 to be performed by the agent 104 in response to the current observation 110. For example, the system 100 may process the current combined embedding 120 using an action selection neural network 122 in accordance with current values of action selection neural network parameters to generate an action selection output 124.
[0078]:
The action selection neural network 122 may be implemented, for example, as a recurrent neural network (e.g., an LSTM) or a feedforward neural network (e.g., a multi-layer perceptron).
(BRI: A system that explicitly uses two distinct embedding generators that combines them to select an action, it fits the main network + sub-NN pattern. The sub-NNs are the embedding modules, and the main network is the action selector)
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, RIM and Hermann.
Agrawal teaches ensemble training on a same training set and deployment on the target hardware.
RIM teaches metadata of hardware.
Hermann teaches adjusting shared parameters.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, RIM and Hermann that reduce the computational resource by updating the auxiliary parameters(Hermann[0060]).
In regards to Claim 3: (Previously Presented)
Agra discloses in:
- the adjusting the parameters of each neural network, is performed by simultaneously adjusting the parameters of each neural network in successive iterations.
[0184]:
in a process of training a single diagnosis assistance neural network model, a plurality of sub-models may be simultaneously trained. The plurality of sub-models may have different layer structures.
[0179]:
Meta-model weights can be adjusted based on observed accuracy at various times during training.
In regards to Claim 4: : (Previously Presented)
Agra does not explicitly disclose:
- the adjusting the parameters of each neural network is performed by adjusting the parameters of each neural network in successive iterations i) until a value of the objective function satisfies a stopping criterion, or ii) for a predetermined number of iterations.
However, RIM discloses in:
- the adjusting the parameters of each neural network is performed by adjusting the parameters of each neural network in successive iterations i) until a value of the objective function satisfies a stopping criterion,
[0182]:
As a result of training a neural network model, optimized parameter values of the model may be obtained. As training of the model using a test data set as described above is repeatedly performed, more appropriate parameter (variable) values may be obtained. When the training is sufficiently performed, optimized values of weights and/or bias may be obtained.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, and RIM.
Agrawal teaches ensemble training on a same training set and deployment on the target hardware.
RIM teaches metadata of hardware.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra and RIM to improve the prediction result can be improved ( RIM [0186]).
In regards to Claim 8: (Previously Presented)
Agra does not explicitly disclose:
- wherein the value of the loss function is computed for each neural network in the group of neural networks: ) based on the difference between the output data generated at the output of each neural network, and the expected output data;
However, RIM discloses in:
- wherein the value of the loss function is computed for each neural network in the group of neural networks: ) based on the difference between the output data generated at the output of each neural network, and the expected output data;
[0189]:
According to an embodiment of the present invention, a plurality of sub-neural network models may obtain the same training data set and individually generate output values. In this case, an ensemble of the plurality of sub-neural network models may be determined as a final neural network model, and parameter values related to each of the plurality of sub-neural network models may be obtained as training results. An output value of the final neural network model may be set to an average value of the output values by the sub-neural network models. Alternatively, in consideration of accuracy obtained as a result of validating each of the sub-neural network models, the output value of the final neural network model may be set to a weighted average value of the output values of the sub-neural network models.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, and RIM.
Agrawal teaches ensemble training on a same training set and deployment on the target hardware.
RIM teaches metadata of hardware.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra and RIM to improve the prediction result can be improved ( RIM [0186]).
In regards to Claim 9: (Previously Presented)
Agra does not explicitly disclose:
- training each neural network in the selected group of neural networks for processing data in the respective hardware environment by inputting second training data into each neural network in the group of neural networks, and adjusting the parameters of each neural network using a second objective function computed based on a difference between output data generated at an output of each neural network and expected output data.
However, RIM discloses in:
- training each neural network in the selected group of neural networks for processing data in the respective hardware environment by inputting second training data into each neural network in the group of neural networks, and adjusting the parameters of each neural network using a second objective function computed based on a difference between output data generated at an output of each neural network and expected output data.
[0173]:
Training of a neural network model may be performed by obtaining a result value using a neural network model to which arbitrary weights are assigned on the basis of training image data, comparing the obtained result value with a label value of the training data, and performing backpropagation according to an error therebetween to optimize the weights,
[0315]:
Each sub-training process may include obtaining result values using a neural network model to which arbitrary weight values are assigned, comparing the obtained result values with label values of training data, and performing backpropagation according to errors therebetween to optimize the weight values.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, and RIM.
Agrawal teaches ensemble training on a same training set and deployment on the target hardware.
RIM teaches metadata of hardware.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra and RIM to improve the prediction result can be improved ( RIM [0186]).
In regards to Claim 10: (Previously Presented)
Agra discloses in:
- the parameters of a lowest neural network in each group of neural networks are shared by all neural networks in the group of neural networks.
[0133]:
For example with tuples 331-333, only 332 is suitable for training both meta-models 351-352 as indicated by the dotted arrows flowing into meta-models 351-352. Multiple meta-models that accept a same shared tuple, such as 332, may take different subsets of (hyperparameter and meta-feature) values that are associated with the shared tuple.
[0083]:
Values 143 and 145 are dissimilar values of hyperparameter 183. For example, hyperparameter 183 may be a count of layers in a neural network, and inference dataset 110 may be a collection of photographs, such that analysis of monochrome photos needs fewer layers than for full-color photos.
In regards to Claim 11: (Previously Presented)
Agra does not explicitly disclose:
- the identifying comprises providing a main neural network and providing each of the one or more sub- neural networks from one or more portions of the main neural network.
However, RIM discloses in:
- the identifying comprises providing a main neural network and providing each of the one or more sub- neural networks from one or more portions of the main neural network.
[0184]:
According to an embodiment of the present invention, in a process of training a single diagnosis assistance neural network model, a plurality of sub-models may be simultaneously trained. The plurality of sub-models may have different layer structures.
[0185]:
In this case, the diagnosis assistance neural network model according to an embodiment of the present invention may be implemented by combining a plurality of sub-neural network models. In other words, training of a neural network model may be performed using an ensemble technique in which a plurality of sub-neural network models are combined.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, and RIM.
Agrawal teaches ensemble training on a same training set and deployment on the target hardware.
RIM teaches metadata of hardware.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra and RIM to improve the prediction result can be improved ( RIM [0186]).
In regards to Claim 14: (Previously Presented)
Agra discloses in:
- ii) performing the repeating until the combined score for the group of neural networks satisfies a predetermined condition.
[0178]:
Boosting may also assign weights to meta-models 651-656 of an ensemble based on their accuracy. Thus, a more reliable meta-model may have more influence over an ensemble score than a less reliable meta-model.
[0179]:
Meta-model weights can be adjusted based on observed accuracy at various times during training. In an embodiment, training ceases when most or all meta-model weights cease to change by at least a threshold amount.
[0133] :
For example with tuples 331-333, only 332 is suitable for training both meta-models 351-352 as indicated by the dotted arrows flowing into meta-models 351-352. Multiple meta-models that accept a same shared tuple, such as 332, may take different subsets of (hyperparameter and meta-feature) values that are associated with the shared tuple.
In regards to Claim 18: (Currently Amended)
Agra discloses in:
- A system for providing a group of neural networks for processing data in a plurality of hardware environments, the system comprising: a first processing system comprising one or more processors configured to carry out a method comprising:
[0183]:
For example, FIG. 7 is a block diagram that illustrates a computer system 700 upon which an embodiment of the invention may be implemented.
[0182] :
According to one embodiment, the techniques described herein are implemented by one or more special-purpose computing devices
[0007]:
FIG. 3 is a block diagram that depicts an example computer that trains an algorithm-specific machine learning ensemble for contextually accurate performance prediction, in an embodiment;
[0113]:
Computer 300 contains configurable trainable algorithms 321-323. Each algorithm is associated with its own machine learning ensemble, such as 390.
[Abstract]:
the algorithm(s) are selected based on the respective scores. Based on the inference dataset, the selected algorithm(s) may be invoked to obtain a result. In an embodiment, the trained regressors are distinctly configured artificial neural networks.
[0129]:
during inferencing after ensemble training, computer 300 may invoke multiple ensembles to detect how well each algorithm would perform for a same unfamiliar inference dataset
- comprising a plurality of parameters and wherein one or more of the parameters of each sub-neural network are shared by the sub-neural network and the main neural network,
such that the parameters of each sub-neural network comprise a respective subset of the plurality of parameters of the main neural network
[0181]:
meta-models 651-656 themselves have hyperparameters (not shown) such as count of layers if an ensemble is composed of neural networks or other hyperparameters if composed of another machine learning algorithm.
[0117]:
Although not shown, ensemble 390 may itself be a composite neural network that contains the neural networks of meta-models 351-352.
(BRI: Perhaps known to the POSITA that by embedding multiple neural networks (meta-models) into a composite network with a final layer for integration, you can create a powerful, flexible, and potentially more accurate ensemble model in which the each meta-model is a separate neural network and the composite meta model is a main neural network)
[0133] :
For example with tuples 331-333, only 332 is suitable for training both meta-models 351-352 as indicated by the dotted arrows flowing into meta-models 351-352. Multiple meta-models that accept a same shared tuple, such as 332, may take different subsets of (hyperparameter and meta-feature) values that are associated with the shared tuple.
[0074] :
In operation, computer 100 obtains inference dataset 110 and should use meta-models, such as 151-153, to select a more or less optimal subset of algorithms 121-123 to eventually be tuned with inference dataset 110. When predicting performance of an algorithm, a meta-model should consider features of the algorithm and features of inference dataset 110.
[0178] :
Boosting may also assign weights to meta-models 651-656 of an ensemble based on their accuracy. Thus, a more reliable meta-model may have more influence over an ensemble score than a less reliable meta-model.
[0179];
Meta-model weights can be adjusted based on observed accuracy at various times during training. In an embodiment, training ceases when most or all meta-model weights cease to change by at least a threshold amount.
[0108] :
After step 204 is sufficiently repeated, all meta-models of all algorithms 121-123 have scores. Based on those scores, at least one promising algorithm is selected for training. For example, computer 100 selects algorithm 122 that has the highest scoring meta-model of all algorithms or the highest mean, median, or modal score of all algorithms.
[0064]:
For each trainable algorithm and each regression meta-model that is respectively associated with the algorithm, a respective score is calculated by invoking the meta-model based on at least one of: a) a respective subset of meta-feature values
0093] :
a subset of meta-feature values as stimulus inputs, shown as dashed arrows entering meta-models 151-153. Training of meta-models is discussed later herein.
[0104] :
most application datasets consist of data units such as pixels, photographs, or tuples (e.g. database table rows). To the extent that a machine learning algorithm may have some configurations that adequately learn with little training data and other configurations that more accurately learn with much training data, one useful meta-feature may be the size of the dataset, such as a count of rows, pixels, or photographs.
[BRI: meta model weights are indeed parameters of a neural network, but they are learned or optimized in a special way to enable faster adaptation and better generalization across tasks . A subset of meta-feature values as stimulus input for the meta-model corresponds to a subset of the parameters of the main neural network, because the meta-features are derived from those parameters)
- computing a performance score for each neural network in the group of neural networks using the adjusted parameters by:
[0096]:
Scores 161-163 share a performance measurement scale. For example, a score may predictively measure how proficient (accuracy such as error rate) would a particular configuration of a particular algorithm become after training for a fixed duration with a particular training dataset, for which inference dataset 110 is representative (e.g. small sample) of the training dataset.
- inputting test data to each neural network, wherein each neural network is implemented in a respective simulation of respective hardware environment;[[,]] executing each neural network in the respective simulations of the respective hardware environments to process the test data;
[0093]:
For example, meta-models 151-153 may each be an already trained neural network that takes a subset of hyperparameter values and a subset of meta-feature values as stimulus inputs, shown as dashed arrows entering meta-models 151-153. Training of meta-models is discussed later herein.
- executing each neural network in the respective simulations of the respective hardware environments to process the test data;
[0091]:
stimulating already-trained meta-models with respective subsets of hyperparameter values and meta-feature values of a new (unfamiliar) inference dataset such as 110, computer 100 may detect how suitable are various hyperparameter configurations of various algorithms 121-123.
[0199] :
OS 810 can execute directly on the bare hardware 820 (e.g., processor(s) 704) of computer system 700. Alternatively, a hypervisor or virtual machine monitor (VMM) 830 may be interposed between the bare hardware 820 and the OS 810.
- and monitoring hardware resource utilization of each neural network when executed in the respective simulations of the respective hardware environments to compute the performance scores
In[ 0201]:
In some instances, the VMM 830 may allow a guest operating system to run as if it is running on the bare hardware 820 of computer system 800 directly. In these instances, the same version of the guest operating system configured to execute on the bare hardware 820 directly may also execute on VMM 830 without modification or reconfiguration. In other words, VMM 830 may provide full hardware and CPU virtualization to a guest operating system in some instances.
[0202]:
In other instances, a guest operating system may be specially designed or configured to execute on VMM 830 for efficiency. In these instances, the guest operating system is “aware” that it executes on a virtual machine monitor. In other words, VMM 830 may provide para-virtualization to a guest operating system in some instances.
repeating the identifying , the inputting, the adjusting, the computing and the generating, for two or more iterations wherein the single objective function is computed at each iteration that represents both the difference between output data generated at the output of each neural network and the expected output data and additionally performance scores of each neural network in the group of neural networks using the adjusted parameter;;
[0106]:
Step 204 is repeated for each trainable algorithm that is available to computer 100. For each meta-model associated with a same algorithm, step 204 calculates a respective score by invoking the meta-model based on at least one of: a) a respective subset of meta-feature values, or b) hyperparameter values of a respective subset of hyperparameters of algorithm.
[0107]:
For example, already-trained distinct meta-models 151-153 may be individually stimulated with a respective subset of meta-feature values 171-174 and a respective subset of hyperparameter values 141-146 as inference inputs. For example, meta-model 151 calculates score 161 based on meta-feature values 171-172 and hyperparameter values 142-143.
selecting from the plurality of groups of neural network generated by the repeating , a group of networks for processing data in the plurality of hardware environments based on the value of the combined score for each group of neural networks
[0098]:
Regardless of score semantics, each meta-model of each algorithm emits a score. Computer 100 may select the best one or few algorithms (perhaps also best hyperparameter values), such as 122 as shown, as ranked based on sorted scores.
- and deploying the selected group of neural networks to a second processing system processing data in the plurality of hardware environments, the selected group of neural networks
[0206]:
Depending on the particular implementation, the precise definition of components or features provided by or within each cloud service layer can vary, but common examples include: Software as a Service (SaaS), in which consumers use software applications that are running upon a cloud infrastructure, while a SaaS provider manages or controls the underlying cloud infrastructure and applications. Platform as a Service (PaaS), in which consumers can use software programming languages and development tools supported by a PaaS provider to develop, deploy, and otherwise control their own applications, while the PaaS provider manages or controls other aspects of the cloud environment (i.e., everything below the run-time execution environment). Infrastructure as a Service (IaaS), in which consumers can deploy and run arbitrary software applications, and/or provision processing, storage, networks, and other fundamental computing resources, while an IaaS provider manages or controls the underlying physical cloud infrastructure
Agra does not explicitly disclose:
- a method comprising: Identifying a group of neural networks including a main neural network and one or more sub-neural networks, each neural network in the group of neural networks comprising a plurality of parameters, the parameters comprising a plurality of weights and biases,
- inputting the same training data into each neural network in the group of neural networks;[[,]]
- adjusting the parameters of each neural network of the group of neural networks based on a backpropagation during a training process that uses a single objective function that represents differences between output data generated at an output of each of the group of networks, and expected output data
generating a combined score for the group of neural networks by combining the performance score of each neural network in the group of neural networks, with a value of a loss function computed for each neural network in the group of neural networks using the adjusted parameters; tied to score
including metadata representing a target hardware environment and/or hardware requirement, of each neural network of the group of neural networks.
However, RIM discloses in:
- the method comprising: Identifying a group of neural networks including a main neural network and one or more sub-neural networks, each neural network in the group of neural networks comprising a plurality of parameters, the parameters comprising a plurality of weights and biases,
[0189]:
an ensemble of the plurality of sub-neural network models may be determined as a final neural network model, and parameter values related to each of the plurality of sub-neural network models may be obtained as training results.
[0189]:
an ensemble of the plurality of sub-neural network models may be determined as a final neural network model, and parameter values related to each of the plurality of sub-neural network models may be obtained as training results.
[0185]:
an embodiment of the present invention may be implemented by combining a plurality of sub-neural network models. In other words, training of a neural network model may be performed using an ensemble technique in which a plurality of sub-neural network models are combined.
[0175]:
A neural network model may be validated using a validation data set. Validation of a neural network model may be performed by obtaining a result value related to a validation data set from a neural network model which has been trained and comparing the result value with a label of the validation data set. The validation may be performed by measuring accuracy of the result value. Parameters of a neural network model (for example, weights and/or bias) or hyperparameters (for example, learning rate) of the neural network model may be adjusted according to a validation result.
- inputting the same training data into each neural network in the group of neural networks;[[,]]
[0189]:
According to an embodiment of the present invention, a plurality of sub-neural network models may obtain the same training data set and individually generate output values. In this case, an ensemble of the plurality of sub-neural network models may be determined as a final neural network model, and parameter values related to each of the plurality of sub-neural network models may be obtained as training results. An output value of the final neural network model may be set to an average value of the output values by the sub-neural network models. Alternatively, in consideration of accuracy obtained as a result of validating each of the sub-neural network models, the output value of the final neural network model may be set to a weighted average value of the output values of the sub-neural network models.
- adjusting the parameters of each neural network of the group of neural networks based on a backpropagation during a training process that uses a single objective function that represents differences between output data generated at an output of each of the group of networks, and expected output data
[0189]:
a plurality of sub-neural network models may obtain the same training data set and individually generate output values. In this case, an ensemble of the plurality of sub-neural network models may be determined as a final neural network model, and parameter values related to each of the plurality of sub-neural network models may be obtained as training results. An output value of the final neural network model may be set to an average value of the output values by the sub-neural network models. Alternatively, in consideration of accuracy obtained as a result of validating each of the sub-neural network models, the output value of the final neural network model may be set to a weighted average value of the output values of the sub-neural network models.
[0173]:
Training of a neural network model may be performed by obtaining a result value using a neural network model to which arbitrary weights are assigned on the basis of training image data, comparing the obtained result value with a label value of the training data, and performing backpropagation according to an error therebetween to optimize the weights,
[0189]:
According to an embodiment of the present invention, a plurality of sub-neural network models may obtain the same training data set and individually generate output values. In this case, an ensemble of the plurality of sub-neural network models may be determined as a final neural network model, and parameter values related to each of the plurality of sub-neural network models may be obtained as training results. An output value of the final neural network model may be set to an average value of the output values by the sub-neural network models. Alternatively, in consideration of accuracy obtained as a result of validating each of the sub-neural network models, the output value of the final neural network model may be set to a weighted average value of the output values of the sub-neural network models.
generating a combined score for the group of neural networks by combining the performance score of each neural network in the group of neural networks, with a value of a loss function computed for each neural network in the group of neural networks using the adjusted parameters;
[0191]:
any one of the plurality of sub-neural network models may be selected on the basis of the accuracy and determined as the final neural network model. A structure of the determined sub-neural network model and parameter values of the determined sub-neural network model obtained as a result of training may be stored.
[0173]:
Training of a neural network model may be performed by obtaining a result value using a neural network model to which arbitrary weights are assigned on the basis of training image data, comparing the obtained result value with a label value of the training data, and performing backpropagation according to an error therebetween to optimize the weights,
[0189]:
According to an embodiment of the present invention, a plurality of sub-neural network models may obtain the same training data set and individually generate output values. In this case, an ensemble of the plurality of sub-neural network models may be determined as a final neural network model, and parameter values related to each of the plurality of sub-neural network models may be obtained as training results. An output value of the final neural network model may be set to an average value of the output values by the sub-neural network models. Alternatively, in consideration of accuracy obtained as a result of validating each of the sub-neural network models, the output value of the final neural network model may be set to a weighted average value of the output values of the sub-neural network models.
including metadata representing a target hardware environment and/or hardware requirement, of each neural network of the group of neural networks.
[0184]:
in a process of training a single diagnosis assistance neural network model, a plurality of sub-models may be simultaneously trained. The plurality of sub-models may have different layer structures.
[0185]:
training of a neural network model may be performed using an ensemble technique in which a plurality of sub-neural network models are combined.
[0162]:
A data set may be obtained from a queue. The data set may be obtained in batches from the queue. For example, when sixty data sets are designated as the size of a batch, sixty data sets may be extracted at a time from the queue. The size of a batch may be limited by the capacity of a RAM of a GPU.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, and RIM.
Agrawal teaches ensemble training on a same training set and deployment on the target hardware.
RIM teaches metadata of hardware.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra and RIM to improve the prediction result can be improved ( RIM [0186]).
Agra and RIM do not explicitly disclose:
- wherein the adjusting comprises simultaneously adjusting the shared parameters for both the main neural network and one or more sub-neural networks
However, Hermann discloses in:
- wherein the adjusting comprises simultaneously adjusting the shared parameters for both the main neural network and one or more sub-neural networks
[0026]:
In some implementations, the temporal autoencoder neural network shares one or more parameters with the action selection neural network, and determining the auxiliary update further includes determining an update to the current values of the shared parameters.
[BRI: in deep reinforcement learning (DRL) systems, temporal autoencoders (often used for feature extraction, denoising, or latent representation learning) and action selection networks (e.g., policy networks) can share parameters and is a common practice in modular or multi-task neural network designs)
[0007]:
the subsystem combines the current observation embedding and the current text embedding to generate a current combined embedding. The subsystem selects, using the current combined embedding, an action to be performed by the agent in response to the current observation.
[0016]:
In some implementations, the current observation embedding is a feature matrix of the current observation, and wherein the current text embedding is a feature vector of the current text string.
[0016] :
In some implementations, the current observation embedding is a feature matrix of the current observation, and wherein the current text embedding is a feature vector of the current text string.
[0040] :
In some implementations, the computing system includes a combining module between the first and second neural network modules and the policy defining neural network module. The combining module has inputs coupled to the first and second neural network modules to combine the environment feature data and the embedded representation of the task input data, and is configured to output combined representation data for the policy defining neural network
[0078]:
At each time step, the system 100 combines the current observation embedding 114 and the current text embedding 118 to determine a current combined embedding 120, and uses the current combined embedding 120 to select an action 102 to be performed by the agent 104 in response to the current observation 110. For example, the system 100 may process the current combined embedding 120 using an action selection neural network 122 in accordance with current values of action selection neural network parameters to generate an action selection output 124.
[0078]:
The action selection neural network 122 may be implemented, for example, as a recurrent neural network (e.g., an LSTM) or a feedforward neural network (e.g., a multi-layer perceptron).
(BRI: A system that explicitly uses two distinct embedding generators that combines them to select an action, it fits the main network + sub-NN pattern. The sub-NNs are the embedding modules, and the main network is the action selector)
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, RIM and Hermann.
Agrawal teaches ensemble training on a same training set and deployment on the target hardware.
RIM teaches metadata of hardware.
Hermann teaches adjusting shared parameters.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, RIM and Hermann that reduce the computational resource by updating the auxiliary parameters(Hermann[0060]).
In regards to Claim 19: (Previously Presented)
Agra discloses:
- computer-implemented method of identifying a neural network for processing data in a hardware environment, the method comprising:
[0183]:
For example, FIG. 7 is a block diagram that illustrates a computer system 700 upon which an embodiment of the invention may be implemented.
In 0182] :
According to one embodiment, the techniques described herein are implemented by one or more special-purpose computing devices
[0007]:
FIG. 3 is a block diagram that depicts an example computer that trains an algorithm-specific machine learning ensemble for contextually accurate performance prediction, in an embodiment;
[0113]:
Computer 300 contains configurable trainable algorithms 321-323. Each algorithm is associated with its own machine learning ensemble, such as 390.
[Abstract]:
the algorithm(s) are selected based on the respective scores. Based on the inference dataset, the selected algorithm(s) may be invoked to obtain a result. In an embodiment, the trained regressors are distinctly configured artificial neural networks.
[0129]:
during inferencing after ensemble training, computer 300 may invoke multiple ensembles to detect how well each algorithm would perform for a same unfamiliar inference dataset
- executing each neural network of the group of neural networks in the hardware environment to process test data;
[0091]:
stimulating already-trained meta-models with respective subsets of hyperparameter values and meta-feature values of a new (unfamiliar) inference dataset such as 110, computer 100 may detect how suitable are various hyperparameter configurations of various algorithms 121-123.
[0199] :
OS 810 can execute directly on the bare hardware 820 (e.g., processor(s) 704) of computer system 700. Alternatively, a hypervisor or virtual machine monitor (VMM) 830 may be interposed between the bare hardware 820 and the OS 810.
- monitoring hardware resource utilization of each neural network when executed in the hardware environment;
[0201]:
In some instances, the VMM 830 may allow a guest operating system to run as if it is running on the bare hardware 820 of computer system 800 directly. In these instances, the same version of the guest operating system configured to execute on the bare hardware 820 directly may also execute on VMM 830 without modification or reconfiguration. In other words, VMM 830 may provide full hardware and CPU virtualization to a guest operating system in some instances.
[0202]:
In other instances, a guest operating system may be specially designed or configured to execute on VMM 830 for efficiency. In these instances, the guest operating system is “aware” that it executes on a virtual machine monitor. In other words, VMM 830 may provide para-virtualization to a guest operating system in some instances.
- computing a performance score for one or more neural networks in the group of neural networks based on the hardware resource utilization and an output of the respective neural network generated
[0096]:
Scores 161-163 share a performance measurement scale. For example, a score may predictively measure how proficient (accuracy such as error rate) would a particular configuration of a particular algorithm become after training for a fixed duration with a particular training dataset, for which inference dataset 110 is representative (e.g. small sample) of the training dataset.
- in response to inputting the test data into the respective neural network and processing the test data with the respective neural network in the hardware environment;
[0093]:
For example, meta-models 151-153 may each be an already trained neural network that takes a subset of hyperparameter values and a subset of meta-feature values as stimulus inputs, shown as dashed arrows entering meta-models 151-153. Training of meta-models is discussed later herein.
- and selecting a neural network from the group of neural networks to process data based on a value of the performance score.
[0064 ]:
One or more of the algorithms are selected based on the respective scores. Based on the inference dataset, the one or more algorithms may be invoked to obtain a result.
Agra does not explicitly disclose:
- and selecting, based on the metadata representing a target hardware environment and/or a hardware requirement, a neural network from the group of neural networks to process data;
However, RIM discloses:
- and selecting, based on the metadata representing a target hardware environment and/or a hardware requirement, a neural network from the group of neural networks to process data;
[0184]:
in a process of training a single diagnosis assistance neural network model, a plurality of sub-models may be simultaneously trained. The plurality of sub-models may have different layer structures.
[0185]:
training of a neural network model may be performed using an ensemble technique in which a plurality of sub-neural network models are combined.
[0162]:
A data set may be obtained from a queue. The data set may be obtained in batches from the queue. For example, when sixty data sets are designated as the size of a batch, sixty data sets may be extracted at a time from the queue. The size of a batch may be limited by the capacity of a RAM of a GPU.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, and RIM.
Agrawal teaches ensemble training and deployment on the target hardware.
RIM teaches metadata of hardware.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, and RIM to improve the prediction result can be improved ( RIM [0186]).
In regards to Claim 20: (Previously Presented)
Agra discloses:
- A non-transitory computer-readable storage medium comprising instructions which when executed by one or more processors cause the one or more processors to carry out the method according to claim 1
[0184], [0135];
Claims 6, 12, 15-17 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over
in view of Sandeep Agrawal et. al. (hereinafter Agra) US 2019/0095756 A1.
in view of Hyung Taek RIM et. al. (hereinafter RIM) US 2019/0221313 A1,
in view of Moritz Karl Hermann et. al. (hereinafter Hermann) US 2021/0110115 A1.
further in view of Mayukh Das et. al. (hereinafter Das) US 2021/0350203 A1 [Foreign Priority: IN 202041019468 Filed 2020-05-05], [in similarity search]
In regards to Claim 6: (Currently Amended)
Agra , RIM and Hermann do not explicitly disclose:
- wherein monitoring hardware resource utilization of each neural network when executed in the respective hardware environments comprises monitoring one or more of:
- a latency when processing the respective neural network using test data in the respective hardware environment;
- a processing utilization in the respective hardware environment when processing the respective neural network using test data;
- a flop count when processing the respective neural network using test data in the respective hardware environment;
- a working memory utilization in the respective hardware environment when processing the respective neural network using test data;
- a memory bandwidth utilization in the respective hardware environment when processing the respective neural network using test data;
- an energy consumption utilization in the respective hardware environment when processing the respective neural network using test data;
- a compression ratio of the respective neural network in the respective hardware environment.
However, Das discloses in:
- a latency when processing the respective neural network in processing test data in the respective hardware environment
[0107]:
In an example, the reward is a collection of different things such as accuracy of a current candidate neural blocks, a device latency, floating point operations per second (FLOPS), a memory consumption, a power consumption and so on.
[0151]:
In an embodiment, the instantiation is converting the abstract DNN (504), that has multiple branches (18B, 18C, 18D), to an actual DNN (i.e. optimized DNN (513)) which is able to perform inference for the intended use case on real data, based on the given target task (501) or task parameters, and target hardware or hardware parameters of the target device (801) on which it is being deployed.
- a processing utilization in the respective hardware environment when processing the respective neural network using test data
[0160]:
the controller (905) evaluates suitability of the best candidate operations (1102) based on a performance, a hardware compatibility, an efficiency and a task precision,
[0008]:
determine hardware parameters of the electronic device used to execute the task based on the performance parameter and the task. The electronic device learns a complete abstract parameterized deep network with multiple possible paths and subsequent instantiation at a deployment time based on the hardware parameters. The abstract parameterized deep network is globally applicable and can be used for learning various ecosystem of electronic devices and diverse tasks. Hence, a time/effort/computing resources used for learning separate pipelines can be saved using the proposed method.
- a flop count when processing the respective neural network in processing test data in the respective hardware environment
[0107]:
In an example, the reward is a collection of different things such as accuracy of a current candidate neural blocks, a device latency, floating point operations per second (FLOPS), a memory consumption, a power consumption and so on.
- a working memory utilization in the respective hardware environment when processing the respective neural network
[0107]:
In an example, the reward is a collection of different things such as accuracy of a current candidate neural blocks, a device latency, floating point operations per second (FLOPS), a memory consumption, a power consumption and so on,
[0151]:
In an embodiment, the instantiation is converting the abstract DNN (504), that has multiple branches (18B, 18C, 18D), to an actual DNN (i.e. optimized DNN (513)) which is able to perform inference for the intended use case on real data, based on the given target task (501) or task parameters, and target hardware or hardware parameters of the target device (801) on which it is being deployed.
- a memory bandwidth utilization in the of the respective hardware environment when processing the respective neural network test data
[0120]:
In another embodiment, the task executor (111) identifies the task to be executed in the electronic device (100). Further, the performance parameter estimator (112) estimates a performance threshold at the time of execution of the identified task. The performance threshold includes an accuracy threshold, a quality threshold of image, a latency threshold, a memory consumption threshold, a power consumption threshold, and a bandwidth threshold.
- an energy consumption utilization in the respective hardware environment when processing the respective neural network using test data
[0107]:
In an example, the reward is a collection of different things such as accuracy of a current candidate neural blocks, a device latency, floating point operations per second (FLOPS), a memory consumption, a power consumption and so on.
[0151]:
In an embodiment, the instantiation is converting the abstract DNN (504), that has multiple branches (18B, 18C, 18D), to an actual DNN (i.e. optimized DNN (513)) which is able to perform inference for the intended use case on real data, based on the given target task (501) or task parameters, and target hardware or hardware parameters of the target device (801) on which it is being deployed
- a compression ratio of the respective neural network in the respective hardware environment
[0015]:
In an embodiment, determining, by the electronic device, the quality of each neural block in the plurality of neural blocks based on the probability distribution in executing the task using the data inputs, the performance parameter and the hardware parameters, includes encoding, by the electronic device, a layer depth and features of neural blocks, creating, by the electronic device, an action space including a set of neural block choices for every learnable block, performing, by the electronic device, a truncation operation to measure usefulness of the set of neural block choices
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, RIM, Hermann and Das.
Agrawal teaches ensemble training on a same training set and deployment on the target hardware.
RIM teaches metadata of hardware.
Hermann teaches adjusting shared parameters.
Das teaches sharing of the sub-neural network and main network.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, RIM, Hermann and Das that can use a single network that can be optimal on all devices and all computing units and learn different models for different devices to get better performance on all devices. Generally, a significant productivity improvement is used to get a desired output (Das[0133]).
In regards to Claim 12: (Previously Presented)
Agra , RIM and Hermann do not explicitly disclose:
- the identifying comprises performing a neural architecture search and/or wherein the identifying comprises maximizing a count of the number of parameters that are shared between the neural networks in the group of neural networks.
However, Das discloses in:
- the identifying comprises performing a neural architecture search and/or wherein the identifying comprises maximizing a count of the number of parameters that are shared between the neural networks in the group of neural networks.
[0139]:
The LazyNAS allows the electronic device (100) to learns a new type of neural model that keeps all different possible branches (18B, 18C, 18D) like a class of multiple models that share commonalities,
[0105]:
a MetaNAS uses predicted latency score from the Metamodel as it is in a
feature space that includes task parameters as well making it general and optimal
simultaneously. Also, a policy-gradient based RL update is used where the expected
reward of a parameterized policy is maximized, argmax.sub.θ(J(θ)=E[r(τ.sub.π{0})]), where
π(θ) is the parameterized policy and τ.sub.π9θ) is the trajectory and r is its reward.
(BRI: It is known in the art that maximizing a count of the number of parameters as a result of RL update for the reward which represents the maximum count of parameter update)
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, RIM, Hermann and Das.
Agrawal teaches ensemble training on a same training set and deployment on the target hardware.
RIM teaches metadata of hardware.
Hermann teaches adjusting shared parameters.
Das teaches sharing of the sub-neural network and main network.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, RIM, Hermann and Das that can use a single network that can be optimal on all devices and all computing units and learn different models for different devices to get better performance on all devices. Generally, a significant productivity improvement is used to get a desired output (Das[0133]).
In regards to Claim 15: (Currently Amended)
Agra discloses in:
- computer-implemented method of identifying a neural network for processing data in a hardware environment, the method comprising:
[0183]:
For example, FIG. 7 is a block diagram that illustrates a computer system 700 upon which an embodiment of the invention may be implemented.
In 0182] :
According to one embodiment, the techniques described herein are implemented by one or more special-purpose computing devices
[0007]:
FIG. 3 is a block diagram that depicts an example computer that trains an algorithm-specific machine learning ensemble for contextually accurate performance prediction, in an embodiment;
[0113]:
Computer 300 contains configurable trainable algorithms 321-323. Each algorithm is associated with its own machine learning ensemble, such as 390.
[Abstract]:
the algorithm(s) are selected based on the respective scores. Based on the inference dataset, the selected algorithm(s) may be invoked to obtain a result. In an embodiment, the trained regressors are distinctly configured artificial neural networks.
[0129]:
during inferencing after ensemble training, computer 300 may invoke multiple ensembles to detect how well each algorithm would perform for a same unfamiliar inference dataset
- executing each neural network of the group of neural networks in the hardware environment to process test data;
[0091]:
stimulating already-trained meta-models with respective subsets of hyperparameter values and meta-feature values of a new (unfamiliar) inference dataset such as 110, computer 100 may detect how suitable are various hyperparameter configurations of various algorithms 121-123.
[0199] :
OS 810 can execute directly on the bare hardware 820 (e.g., processor(s) 704) of computer system 700. Alternatively, a hypervisor or virtual machine monitor (VMM) 830 may be interposed between the bare hardware 820 and the OS 810.
- monitoring hardware resource utilization of each neural network when executed in the hardware environment;
[ 0201]:
In some instances, the VMM 830 may allow a guest operating system to run as if it is running on the bare hardware 820 of computer system 800 directly. In these instances, the same version of the guest operating system configured to execute on the bare hardware 820 directly may also execute on VMM 830 without modification or reconfiguration. In other words, VMM 830 may provide full hardware and CPU virtualization to a guest operating system in some instances.
[0202]:
In other instances, a guest operating system may be specially designed or configured to execute on VMM 830 for efficiency. In these instances, the guest operating system is “aware” that it executes on a virtual machine monitor. In other words, VMM 830 may provide para-virtualization to a guest operating system in some instances.
- computing a performance score for one or more neural networks in the group of neural networks based on the hardware resource utilization and an output of the respective neural network generated
[0096]:
Scores 161-163 share a performance measurement scale. For example, a score may predictively measure how proficient (accuracy such as error rate) would a particular configuration of a particular algorithm become after training for a fixed duration with a particular training dataset, for which inference dataset 110 is representative (e.g. small sample) of the training dataset.
- in response to inputting the test data into the respective neural network and processing the test data with the respective neural network in the hardware environment;
[0093]:
For example, meta-models 151-153 may each be an already trained neural network that takes a subset of hyperparameter values and a subset of meta-feature values as stimulus inputs, shown as dashed arrows entering meta-models 151-153. Training of meta-models is discussed later herein.
- and selecting a neural network from the group of neural networks to process data based on a value of the performance score.
[0064 ]:
One or more of the algorithms are selected based on the respective scores. Based on the inference dataset, the one or more algorithms may be invoked to obtain a result.
Agra does not explicitly disclose:
- and selecting, based on the metadata representing a target hardware environment and/or a hardware requirement, a neural network from the group of neural networks to process data;
However, RIM discloses:
- and selecting, based on the metadata representing a target hardware environment and/or a hardware requirement, a neural network from the group of neural networks to process data;
[0184]:
in a process of training a single diagnosis assistance neural network model, a plurality of sub-models may be simultaneously trained. The plurality of sub-models may have different layer structures.
[0185]:
training of a neural network model may be performed using an ensemble technique in which a plurality of sub-neural network models are combined.
[0162]:
A data set may be obtained from a queue. The data set may be obtained in batches from the queue. For example, when sixty data sets are designated as the size of a batch, sixty data sets may be extracted at a time from the queue. The size of a batch may be limited by the capacity of a RAM of a GPU.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, and RIM.
Agrawal teaches ensemble training and deployment on the target hardware.
RIM teaches metadata of hardware.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, and RIM to improve the prediction result can be improved ( RIM [0186]).
In regards to Claim 16: (Previously Presented)
Agra , RIM and Hermann do not explicitly disclose:
- processing input data with the selected neural network in the hardware environment, and dynamically shifting a processing of the input data by the neural network between a plurality of processors of the hardware environment responsive a performance score computed for the processing meeting a specified condition.
However, Das discloses in:
- processing input data with the selected neural network in the hardware environment, and dynamically shifting a processing of the input data by the neural network between a plurality of processors of the hardware environment responsive a performance score computed for the processing meeting a specified condition.
[0129]:
Instantiation is the key step in the dynamic deployment step. Dynamic deployment step includes a dynamic deployment of the abstract network (504) onto the target devices via an information maximization given hardware/task parameters,
[0008 ]:
determine hardware parameters of the electronic device used to execute the task based on the performance parameter and the task. The electronic device learns a complete abstract parameterized deep network with multiple possible paths and subsequent instantiation at a deployment time based on the hardware parameters.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, RIM, Hermann and Das.
Agrawal teaches ensemble training on a same training set and deployment on the target hardware.
RIM teaches metadata of hardware.
Hermann teaches adjusting shared parameters.
Das teaches sharing of the sub-neural network and main network.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, RIM, Hermann and Das that can use a single network that can be optimal on all devices and all computing units and learn different models for different devices to get better performance on all devices. Generally, a significant productivity improvement is used to get a desired output (Das[0133]).
In regards to Claim 17: (Previously Presented)
Agra , RIM and Hermann do not explicitly disclose:
- the computer-implemented method according to claim 1, wherein the identifying a group of neural networks comprises: I) performing a neural architecture search; or ii) performing a differential neural architecture search; and wherein the computing a performance score for each neural network in the group of neural networks, comprises approximating a performance score for each neural network in the group of neural networks for the respective hardware environment using a differentiable performance model for each neural network.
However, Das discloses in:
- the identifying a group of neural networks comprises: i) performing a neural architecture search;
[0006]:
The principal object of the embodiments herein is to provide a NAS method and an electronic device for generating an optimized DNN model to execute a task,
[0052]:
the NAS is proposed as a solution for automatically determining the correct choice of neural blocks,
[0129]:
Instantiation is the key step in the dynamic deployment step. Dynamic deployment step includes a dynamic deployment of the abstract network (504) onto the target devices via an information maximization given hardware/task parameters,
[0008 ]:
determine hardware parameters of the electronic device used to execute the task based on the performance parameter and the task.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, RIM, Hermann and Das.
Agrawal teaches ensemble training on a same training set and deployment on the target hardware.
RIM teaches metadata of hardware.
Hermann teaches adjusting shared parameters.
Das teaches sharing of the sub-neural network and main network.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, RIM, Hermann and Das that can use a single network that can be optimal on all devices and all computing units and learn different models for different devices to get better performance on all devices. Generally, a significant productivity improvement is used to get a desired output (Das[0133]).
In regards to Claim 21: (Previously Presented)
Agra , RIM and Hermann do not explicitly disclose:
- selecting a neural network from the selected group of neural network to process data for a target hardware environment and processing input data with the selected neural network in the target hardware environment
However, Das discloses in:
- selecting a neural network from the selected group of neural network to process data for a target hardware environment and processing input data with the selected neural network in the target hardware environment
[0114]:
In an embodiment, the optimal DNN model generator (114) represents the intermediate DNN model using the plurality of neural blocks. Further, the optimal DNN model generator (114) provides the data inputs to the intermediate DNN model. Further, the optimal DNN model generator (114) determines the quality of each neural block in the plurality of neural blocks based on the probability distribution in executing the task using the data inputs, the performance parameter and the hardware parameters. Further, the optimal DNN model generator (114) selects the optimal neural blocks from the plurality of neural blocks based on the quality of each neural block,
[0065]:
The NAS controller (110) is configured to determine hardware parameters (also called as hardware configuration) of the electronic device (100) for executing the task based on the performance parameter and the task. Examples for the hardware parameters are, but not limited to a processor speed, number of cores in the processor (130), a data transmission speed wireless module, a storage capacity of the memory.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, RIM, Hermann and Das.
Agrawal teaches ensemble training on a same training set and deployment on the target hardware.
RIM teaches metadata of hardware.
Hermann teaches adjusting shared parameters.
Das teaches sharing of the sub-neural network and main network.
It would have obvious to one of ordinary skill in the art before the effective filing date of
the present application to combine Agra, RIM, Hermann and Das that can use a single network that can be optimal on all devices and all computing units and learn different models for different devices to get better performance on all devices. Generally, a significant productivity improvement is used to get a desired output (Das[0133]).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TIRUMALE KRISHNASWAMY RAMESH whose telephone number is (571)272-4605. The examiner can normally be reached by phone.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li B Zhen can be reached on phone (571-272-3768). The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit:
https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TIRUMALE K RAMESH/Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights