DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements filed 02/17/2026, 04/30/2026, and 08/14/2026 have been considered.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 01/14/2026 has been entered.
Response to Amendment
The amendments filed 01/14/2026 has been entered. The following action is in response to the amendment/remarks of 01/14/2026.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 5, 9-15, and 27-29 are rejected under 35 U.S.C. 103 as being unpatentable over Cataltepe United States Patent Application Publication US 2019/0279102 in view of Kolar United States Patent Application Publication US 2021/0304061 in further view of Ostergaard United States Patent Application Publication US 2022/0277231.
Regarding claim 1, Cataltepe discloses One or more non-transitory computer readable media (NTCRM) comprising instructions for predicting performance metrics for machine learning (ML) models, wherein the instructions one or more processors to:
generate a group of ML models in a model development environment (Cataltepe, abstract, online machine learning system; Cataltepe, para [0130-131], with regards to fig 3A each model within online machine learning engine 305 generated);
operate a proxy feedback engine (Cataltepe, para [0264], OES can be used as a proxy for feedback to the online machine learning system);
train a portion of ML models from the group of ML models to obtain one or more actual performance metrics for corresponding ML models in the portion of ML models (Cataltepe, para [0145], trains new models, where a new model is interpreted as a subset of the existing models);
operate a semi-supervised learning (SSL) mechanism to learn relationships between the one or more actual performance metrics and the corresponding ML models in the portion of ML models (Cataltepe, para [0076], with regards to fig 3B and 4, online human expert feedback system OEFBS),
the learned relationships corresponding to a mapping between the one or more actual performance metrics and the corresponding ML models, wherein the one or more actual performance metrics include accuracy, latency, and power consumption (Cataltepe, para [0116] teaches the learned relationships corresponding to a mapping between the one or more actual performance metrics such as accuracy and delays, i.e. latency).
Cataltepe does not disclose:
for each ML model in the set of ML models, operate a proxy feedback engine to generate a proxy score for a corresponding ML model in the group of ML models, wherein the proxy score of the corresponding ML model is based on one or more predicted performance metrics for the corresponding ML model, the one or more predicted performance metrics including a model performance estimation score associated with a type of hardware platform on which the corresponding ML model is to be deployed;
update the proxy feedback engine with the learned relationships for generating proxy scores for another set of ML models,
and provide, based on the proxy scores for the group of ML models, a trained ML model for deployment on a device including the type of hardware platform.
Kolar discloses:
for each ML model in the set of ML models, operate a proxy feedback engine to generate a proxy score for a corresponding ML model in the set of ML models, wherein the proxy score of the corresponding ML model is based on one or more predicted performance metrics for the corresponding ML model (Kolar, para [0079-83], ‘Employ a simple, multi-class classifier to predict A.sub.i from A.sub.j. The classifier may input just one input A.sub.j (if A.sub.j is also categorical, then a one-hot encoding A.sub.j can be considered features) and predict the output class representing the possible values for A.sub.i. The model is trained from the dataset, and some accuracy metric (say, AUC or Precision) is measured. A high score indicates that there is some function that can map A.sub.i to A.sub.j.’);
update the proxy feedback engine with the learned relationships for generating proxy scores for another set of ML models (Kolar, para [0084], ‘each entry 532 in proxy feature mapping database 516 may be of the form <feature, proxy-feature, weight, reason>. Note that the weight can be based on the ones computed as above (e.g., using the scores computed for the mappings). Later, when the user interacts with interpretability module 502, this score will be suitably changed based on user preferences to map feature A.sub.i to an equivalent, understandable feature A.sub.j’),
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the learning to include the steps of the feedback engine. The motivation for doing so would have been provide additional feedback and ability to update for users (Kolar, para [0003-4]).
Cataltepe in view of Kolar does not explicitly disclose:
the one or more predicted performance metrics including a model performance estimation score associated with a type of hardware platform on which the corresponding ML model is to be deployed;
and provide, based on the proxy scores for the group of ML models, a trained ML model for deployment on a device including the type of hardware platform.
Ostergaard discloses:
the one or more predicted performance metrics including a model performance estimation score associated with a type of hardware platform on which the corresponding ML model is to be deployed (Ostergaard, para [0032-33], monitor the data for criteria that indicate that the subject type or situation type of the machine learning inference has changed; Ostergaard, para [0035], comparison analysis of collected vs expected data for each available data type and source to look at a sum deviancy level, representing a score);
and provide, based on the proxy scores for the group of ML models, a trained ML model for deployment on a device including the type of hardware platform (Ostergaard, fig 2, machine learning model orchestration system, representing a proxy, distributes determined optimal models to devices; Ostergaard, para [0055], following a determination to refresh models, they update distribution system provides corresponding edge computers with model updates).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the edge network to include the invocation based on system hardware based on the teachings of Ostergaard. The motivation for doing so would have been to provide automatic updates (Ostergaard, para [0007]).
Regarding claim 5, Cataltepe in view of Kolar in further view of Ostergaard discloses the one or more NTCRM of claim 1. Cataltepe additionally discloses wherein the individual ML models are generated for the respective users based on one or more inputs provided by the respective users (Cataltepe, para [0076-77], with regards to fig 3B online human expert feedback 327).
Regarding claim 9, Cataltepe in view of Kolar in further view of Ostergaard discloses the one or more NTCRM of claim 1. Cataltepe additionally discloses wherein the SSL mechanism comprises an inductive SSL method or a transductive SSL method (Cataltepe, para [0103], labels are available on demand or some of the time (semi-supervised learning)).
Regarding claim 10, Cataltepe in view of Kolar in further view of Ostergaard discloses the one or more NTCRM of claim 1. Cataltepe additionally discloses wherein the SSL mechanism comprises an SSL method selected from a group including a self-training wrapper method, a co-training wrapper method, a boosting wrapper method, an unsupervised pre-processing feature extraction method, an unsupervised pre-processing pre-training method, an unsupervised pre-processing cluster-then- label method, and an intrinsically semi-supervised method, a graph convolutional network (GCN), and a transductive support vector machine (TSVM) (Cataltepe, para [0103], labels are available on demand or some of the time (semi-supervised learning)).
Regarding claim 11, Cataltepe discloses an apparatus for providing a machine learning model development environment (MDE), the apparatus comprising:
interface circuitry to obtain machine learning (MVL) inputs from a client device (Cataltepe, para [0076], online expert feedback system); and
processor circuitry communicatively coupled with the interface circuitry, wherein the processor circuitry is, in response to receipt of an individual input of the ML inputs from the client device, to: obtain an ML model based on the individual input wherein (Cataltepe, Fig. 1):
at least one of the ML model is used by a semi-supervised learning (SSL) mechanism to learn relationships between actual performance metrics and trained ML models (Cataltepe, para [0076], with regards to fig 3B and 4, online human expert feedback system OEFBS),
the learned relationships corresponding to a mapping between the one or more actual performance metrics and the corresponding ML models, wherein the one or more actual performance metrics include accuracy, latency, and power consumption (Cataltepe, para [0116] teaches the learned relationships corresponding to a mapping between the one or more actual performance metrics such as accuracy and delays, i.e. latency);
and
provide the ML model to the client device via the interface circuitry (Cataltepe, Fig. 1 and para [0077] teaches providing the model to client devices via interface circuitry).
Cataltepe does not disclose:
for each ML model in the set of ML models, operate a proxy feedback engine to generate a proxy score for a corresponding ML model in the set of ML models, wherein the proxy score of the corresponding ML model is based on one or more predicted performance metrics for the corresponding ML model
update the proxy feedback engine with the learned relationships for generating proxy scores for another set of ML models.
Kolar discloses:
obtain an ML model based on the individual input, obtain a proxy score for the ML model from a proxy feedback engine, wherein:
the proxy score for the ML model is based on a set of predicted performance metrics for the ML model, at least one of the ML model and the proxy score is used by a semi-supervised learning (SSL) mechanism to learn relationships between actual performance metrics and trained ML models (Kolar, para [0079-83], ‘Employ a simple, multi-class classifier to predict A.sub.i from A.sub.j. The classifier may input just one input A.sub.j (if A.sub.j is also categorical, then a one-hot encoding A.sub.j can be considered features) and predict the output class representing the possible values for A.sub.i. The model is trained from the dataset, and some accuracy metric (say, AUC or Precision) is measured. A high score indicates that there is some function that can map A.sub.i to A.sub.j.’);
the learned relationships are used by the proxy feedback engine for generating proxy scores for other ML models and provide the ML model and the proxy score to the client device via the interface circuitry (Kolar, para [0084], ‘each entry 532 in proxy feature mapping database 516 may be of the form <feature, proxy-feature, weight, reason>. Note that the weight can be based on the ones computed as above (e.g., using the scores computed for the mappings). Later, when the user interacts with interpretability module 502, this score will be suitably changed based on user preferences to map feature A.sub.i to an equivalent, understandable feature A.sub.j’).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the learning to include the steps of the feedback engine. The motivation for doing so would have been provide additional feedback and ability to update for users (Kolar, para [0003-4]).
Cataltepe in view of Kolar does not explicitly disclose:
obtain an ML model based on the individual input, the individual input including a type of hardware platform for operating the ML model,obtain a proxy score for the ML model from a proxy feedback engine, wherein the proxy score for the ML model is based on a set group of predicted performance metrics for the ML model, the predicted performance metrics including a model performance estimation score associated with the type of hardware platform on which the ML model is to be deployed, train ML models to obtain actual performance metrics, the proxy score mapped to the actual performance metrics based on a semi-supervised learning (SSL) mechanism.
Ostergaard discloses:
obtain an ML model based on the individual input, the individual input including a type of hardware platform for operating the ML model, obtain a proxy score for the ML model from a proxy feedback engine, wherein the proxy score for the ML model is based on a set group of predicted performance metrics for the ML model, the predicted performance metrics including a model performance estimation score associated with the type of hardware platform on which the ML model is to be deployed, train ML models to obtain actual performance metrics, the proxy score mapped to the actual performance metrics based on a semi-supervised learning (SSL) mechanism (Ostergaard, para [0032-33], monitor the data for criteria that indicate that the subject type or situation type of the machine learning inference has changed; Ostergaard, para [0035], comparison analysis of collected vs expected data for each available data type and source to look at a sum deviancy level, representing a score; Ostergaard, fig 2, machine learning model orchestration system, representing a proxy, distributes determined optimal models to devices; Ostergaard, para [0055], following a determination to refresh models, they update distribution system provides corresponding edge computers with model updates).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the edge network to include the invocation based on system hardware based on the teachings of Ostergaard. The motivation for doing so would have been to provide automatic updates (Ostergaard, para [0007]).
Regarding claim 12, Cataltepe in view of Kolar in further view of Ostergaard discloses the apparatus of claim 11. Cataltepe additionally discloses wherein the processor circuitry is further to: operate a server-side MDE application to enable receipt of the ML inputs from the client device; and provide the ML model and the proxy score to be rendered within a client-side MDE application (Cataltepe, para [0082], awaits receipt from human input/feedback).
Regarding claim 13, Cataltepe in view of Kolar in further view of Ostergaard discloses the apparatus of claim 11. Cataltepe additionally discloses wherein the processor circuitry is, in response to receipt of the individual input of the ML inputs from the client device, further to: obtain, from an ML operations library, a set of ML operations compatible with the ML model; and provide, via the interface circuitry, the set of MVL operations to be rendered within the client-side MDE application (Cataltepe, para [0082], in response to the receipt, perform learning functions; Cataltepe, para [0073], with regards to fig 3A, multiple machine learning algorithms/models used in online machine learning system).
Regarding claim 14, Cataltepe in view of Kolar in further view of Ostergaard discloses the apparatus of claim 11. Kolar additionally discloses wherein the ML inputs include an ML task and an ML domain, and a proxy function used by the proxy feedback engine to produce the proxy score is based on the ML task and the ML domain (Kolar, para [0079-83], ‘Employ a simple, multi-class classifier to predict A.sub.i from A.sub.j. The classifier may input just one input A.sub.j (if A.sub.j is also categorical, then a one-hot encoding A.sub.j can be considered features) and predict the output class representing the possible values for A.sub.i. The model is trained from the dataset, and some accuracy metric (say, AUC or Precision) is measured. A high score indicates that there is some function that can map A.sub.i to A.sub.j.’).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the learning to include the steps of the feedback engine. The motivation for doing so would have been provide additional feedback and ability to update for users (Kolar, para [0003-4]).
Regarding claim 15, Cataltepe in view of Kolar in further view of Ostergaard discloses the apparatus of claim 14. Kolar additionally discloses wherein the ML inputs further include hardware platform specifications, and the proxy function used by the proxy feedback engine is further based on the hardware platform specifications (Kolar, para [0013-14], network hardware configurations).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the learning to include the steps of the feedback engine. The motivation for doing so would have been provide additional feedback and ability to update for users (Kolar, para [0003-4]).
Regarding claim 27, Cataltepe in view of Kolar in further view of Ostergaard discloses the one or more NTCRM of claim 1. Ostergaard additionally discloses wherein the hardware platform is associated with at least one of an Internet-of-Things (IoT) device or an autonomous sensor (Ostergaard, para [0058], IoT network).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the edge network to include the invocation based on system hardware based on the teachings of Ostergaard. The motivation for doing so would have been to provide automatic updates (Ostergaard, para [0007]).
Regarding claim 28, Cataltepe in view of Kolar in further view of Ostergaard discloses the one or more NTCRM of claim 1. Kolar additionally discloses wherein execution of the instructions is to cause the computing system to send a mini-batch of data through a portion of untrained ML models to generate activations within the untrained ML models to use as the proxy scores (Kolar, para [0033], semi-supervised mlm).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the learning to include the steps of the feedback engine. The motivation for doing so would have been provide additional feedback and ability to update for users (Kolar, para [0003-4]).
Regarding claim 29, Cataltepe in view of Kolar in further view of Ostergaard discloses the one or more NTCRM of claim 1. Cataltepe additionally discloses wherein the instructions cause the one or more processors to identify the learned relationships based on an encoding of pair-wise similarities with weighted edges (Cataltepe, para [0259] teaches identifying learned relationships based on a weighted encoding of pairwise similarities using Euclidean distances).
Claim(s) 2-4, 6-8 are rejected under 35 U.S.C. 103 as being unpatentable over Cataltepe United States Patent Application Publication US 2019/0279102 in view of Kolar United States Patent Application Publication US 2021/0304061 in further view of Ostergaard United States Patent Application Publication US 2022/0277231 as modified by Amizadeh United States Patent Application Publication US 2019/0347548.
Regarding claim 2, Cataltepe in view of Kolar in further view of Ostergaard discloses the one or more NTCRM of claim 1. Cataltepe in view of Kolar in further view of Ostergaard does not disclose the additional limitations of claim 2.
Amizadeh discloses wherein the operation of the proxy feedback engine takes place without training the ML models in the set of ML models (Amizadeh, para [0025-26], metrics, i.e. scores, generated for each untrained models).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the system to include selections of functions based on the teachings of Amizadeh. The motivation for doing so would have been to efficiently solve machine learning problems (Amizadeh, para [0001]).
Regarding claim 3, Cataltepe in view of Kolar in further view of Ostergaard discloses the one or more NTCRM of claim 1. Cataltepe in view of Kolar in further view of Ostergaard does not disclose the additional limitations of claim 3.
Amizadeh discloses wherein, to operate the proxy feedback engine, execution of the instructions is to cause the computing system to: select a first proxy function from among a plurality of proxy functions for a first ML model in the set of ML models; operate the first proxy function to produce a first proxy score for the first ML model; select a second proxy function from among a plurality of proxy functions for a second ML model in the set of ML models; and operate the second proxy function to produce a second proxy score for the first ML model (Amizadeh, para [0025-27], with regards to fig 2, elements 230-240. metrics, i.e. scores, generated for each untrained models, representing a first model and a second model used for selection; Amizadeh, para [028], with regards to fig 2, element 250, provides output).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the system to include selections of functions based on the teachings of Amizadeh. The motivation for doing so would have been to efficiently solve machine learning problems (Amizadeh, para [0001]).
Regarding claim 4, Cataltepe in view of Kolar in further view of Ostergaard discloses the one or more NTCRM of claim 1. Cataltepe discloses weighted averages (Cataltepe, para [0174], weighted average of the inputs).
Cataltepe in view of Kolar in further view of Ostergaard does not disclose the additional limitations of claim 4.
Amizadeh discloses wherein, to operate the proxy feedback engine, execution of the instructions is to cause the computing system to: select, for at least one ML model in the set of ML models, two or more proxy functions from among a plurality of proxy functions; individually operate each of the two or more proxy functions to produce respective proxy scores for the at least one ML model; and determine a final proxy score for the at least one ML model based on a combination of the respective proxy scores using ensemble averaging of the respective proxy scores, or using an ensemble learning method based on the respective proxy scores (Amizadeh, para [0025-27], with regards to fig 2, elements 230-240. metrics, i.e. scores, generated for each untrained models, representing a first model and a second model used for selection; Amizadeh, para [028], with regards to fig 2, element 250, provides output).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the system to include selections of functions based on the teachings of Amizadeh. The motivation for doing so would have been to efficiently solve machine learning problems (Amizadeh, para [0001]).
Regarding claim 6, Cataltepe in view of Kolar in further view of Ostergaard discloses the one or more NTCRM of claim 5. Cataltepe in view of Kolar in further view of Ostergaard does not disclose the additional limitations of claim 6.
Amizadeh discloses wherein the one or more inputs provided by the respective users include an ML task in an ML domain, and wherein, to operate the proxy feedback engine, execution of the instructions is to cause the computing system to: select, for each ML model in the set of ML models, a proxy function from a plurality of proxy functions based on the ML task and the ML domain provided by the respective users (Amizadeh, para [0025-27], with regards to fig 2, elements 230-240. metrics, i.e. scores, generated for each untrained models, representing a first model and a second model used for selection; Amizadeh, para [028], with regards to fig 2, element 250, provides output).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the system to include selections of functions based on the teachings of Amizadeh. The motivation for doing so would have been to efficiently solve machine learning problems (Amizadeh, para [0001]).
Regarding claim 7, Cataltepe in view of Kolar in further view of Ostergaard in further view of Amizadeh discloses the one or more NTCRM of claim 6.
Kolar additionally discloses wherein the one or more inputs provided by the respective users further include hardware platform specifications, and the selection of the proxy function for each ML model is further based on the hardware platform specifications (Kolar, para [0013-14], network hardware configurations).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the learning to include the steps of the feedback engine. The motivation for doing so would have been provide additional feedback and ability to update for users (Kolar, para [0003-4]).
Regarding claim 8, Cataltepe in view of Kolar in further view of Ostergaard in further view of Amizadeh discloses the one or more NTCRM of claim 3.
Cataltepe additionally discloses wherein the plurality of proxy functions include one or more of associative arrays, mapping functions, dictionaries, hash tables, look-up tables (LUTs), linked lists, ML classifiers, parameter counting, computational throughput metrics, Jacobian covariance functions, saliency pruning functions, channel pruning functions, heuristic functions, and hyper-heuristic functions (Cataltepe, para [0139], stacking classifier).
Claim(s) 17-22 are rejected under 35 U.S.C. 103 as being unpatentable over Cataltepe United States Patent Application Publication US 2019/0279102 in view of Amizadeh United States Patent Application Publication US 2019/0347548 in further view of Ostergaard United States Patent Application Publication US 2022/0277231.
Regarding claim 17, Cataltepe discloses an apparatus employed as a proxy feedback engine of a machine learning (ML) computing system, the apparatus comprising:
interface circuitry to obtain ML model configurations (MLMCs) via an ML model development environment (MMDE); and
processor circuitry communicatively coupled with the interface circuitry, wherein the processor circuitry is, for an individual MLMC of the obtained MLMCs (Cataltepe, para [0145], trains new models, where a new model is interpreted as a subset of the existing models).
the trained ML model to obtain actual performance metrics (Cataltepe, para [0145], trains new models, to obtain actual performance metrics);
… performance metrics based on a semi-supervised learning (SSL) mechanism to learn relationships between the actual performance metrics and the trained ML model (Cataltepe, para [0076], with regards to fig 3B and 4, online human expert feedback system OEFBS),
the learned relationships corresponding to a mapping between the one or more actual performance metrics and the corresponding ML models, wherein the one or more actual performance metrics include accuracy, latency, and power consumption (Cataltepe, para [0116] teaches the learned relationships corresponding to a mapping between the one or more actual performance metrics such as accuracy and delays, i.e. latency).
Cataltepe does not disclose:
select one or more proxy functions from a set of proxy functions based on information in the individual MLMC,
determine a proxy score for an ML model corresponding to the individual ML-MC using the selected one or more proxy functions, wherein the proxy score is based on a set of predicted performance metrics for the ML model, provide the determined proxy score to a model analysis engine via the interface circuitry, and obtain, from the model analysis engine, an updated mapping of actual ML performance metrics to ML models based at least in part on the determined proxy score for generating updated proxy scores,
the proxy score mapped to the actual performance metrics.
Amizadeh discloses processor circuitry to:
select one or more proxy functions from a set of proxy functions based on information in the individual MLMC (Amizadeh, para [24-27], selection module selects functions),
determine a proxy score for an ML model corresponding to the individual ML-MC using the selected one or more proxy functions, wherein the proxy score is based on a set of predicted performance metrics for the ML model, provide the determined proxy score to a model analysis engine via the interface circuitry, and obtain, from the model analysis engine, an updated mapping of actual ML performance metrics to ML models based at least in part on the determined proxy score for generating updated proxy scores (Amizadeh, para [0025-27], with regards to fig 2, elements 230-240. metrics, i.e. scores, generated for each untrained models, representing a first model and a second model used for selection; Amizadeh, para [028], with regards to fig 2, element 250, provides output),
the proxy score mapped to the actual performance metrics (Amizadeh, para [0025-27], and fig 2, elements 230-240).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the system to include selections of functions based on the teachings of Amizadeh. The motivation for doing so would have been to efficiently solve machine learning problems (Amizadeh, para [0001]).
Cataltepe in view of Amizadeh does not explicitly disclose:
the predicted performance metrics including a model performance estimation score associated with a type of hardware platform on which the ML model is to be deployed;
and provide, based on the updated mapping of actual ML performance metrics, a trained ML model for deployment on a device including the type of hardware platform.
Ostergaard discloses:
the predicted performance metrics including a model performance estimation score associated with a type of hardware platform on which the ML model is to be deployed (Ostergaard, para [0032-33], monitor the data for criteria that indicate that the subject type or situation type of the machine learning inference has changed; Ostergaard, para [0035], comparison analysis of collected vs expected data for each available data type and source to look at a sum deviancy level, representing a score);
and provide, based on the updated mapping of actual ML performance metrics, a trained ML model for deployment on a device including the type of hardware platform (Ostergaard, fig 2, machine learning model orchestration system, representing a proxy, distributes determined optimal models to devices; Ostergaard, para [0055], following a determination to refresh models, they update distribution system provides corresponding edge computers with model updates).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the edge network to include the invocation based on system hardware based on the teachings of Ostergaard. The motivation for doing so would have been to provide automatic updates (Ostergaard, para [0007]).
Regarding claim 18, Cataltepe in view of Amizadeh in further view of Ostergaard discloses the apparatus of claim 17. Amizadeh additionally discloses wherein the individual ML-MC includes an ML task and an ML domain, and the selection of the proxy function is based on the ML task and the ML domain (Amizadeh, para [0027], selection based on candidate networks).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the system to include selections of functions based on the teachings of Amizadeh. The motivation for doing so would have been to efficiently solve machine learning problems (Amizadeh, para [0001]).
Regarding claim 19, Cataltepe in view of Amizadeh in further view of Ostergaard discloses the apparatus of claim 17. Cataltepe additionally discloses wherein the proxy score includes an ML model performance component including one or more predicted performance metrics related to operation of the ML model (Cataltepe, para [0242], “While training the autoencoder, we compute the error E.sub.k for the instances in each class k. If the error for class k is much larger than the others, then we oversample the instances in class k while training the autoencoder” autoencoder uses mean absolute error (MAE)).
Regarding claim 20, Cataltepe in view of Amizadeh in further view of Ostergaard discloses the apparatus of claim 19. Cataltepe additionally discloses wherein the one or more predicted performance metrics of the ML model performance component include one or more of accuracy, precision, negative predictive value (NPV), recall, specificity, false positive rate, false negative rate, F score, markedness, receiver operating characteristic (ROC), area under the ROC curve (AUC), an error value, mean absolute error (MAE), mean reciprocal rank (MRR), mean squared error (MSE), root MSE (RMSE), correlation coefficient (R), coefficient of determination (R ), cumulative gain (CG), discounted CG (DCG), normalized DCG (NDCG), Intersection over Union (IoU), perplexity, Wasserstein metric, Frechet inception distance (FID), and Damerau-Levenshtein distance (Cataltepe, para [0242], “While training the autoencoder, we compute the error E.sub.k for the instances in each class k. If the error for class k is much larger than the others, then we oversample the instances in class k while training the autoencoder” autoencoder uses mean absolute error (MAE)).
Regarding claim 21, Cataltepe in view of Amizadeh in further view of Ostergaard discloses the apparatus of claim 18. Cataltepe additionally discloses wherein the individual ML-MC further includes hardware platform specifications, and the selection of the proxy function is further based on the hardware platform specifications (Cataltepe, para [0116], throughput measured).
Regarding claim 22, Cataltepe in view of Amizadeh in further view of Ostergaard discloses the apparatus of claim 21. Cataltepe additionally discloses wherein the proxy score includes a platform performance component including one or more predicted performance metrics performance of a hardware platform that is to operate the ML model (Cataltepe, para [0116], throughput measured).
Claim(s) 16 is rejected under 35 U.S.C. 103 as being unpatentable over Cataltepe United States Patent Application Publication US 2019/0279102 in view of Kolar United States Patent Application Publication US 2021/0304061 in further view of Ostergaard United States Patent Application Publication US 2022/0277231 as modified by Shen United States Patent Application Publication US 2022/0180125.
Regarding claim 16, Cataltepe in view of Kolar in further view of Ostergaard discloses the apparatus of claim 11. Cataltepe in view of Kolar in further view of Ostergaard does not disclose the additional limitations of claim 16.
Shen discloses wherein the interface circuitry is further to: obtain a training command from the client device (Shen, para [0215-216], client 1200 provides updated map request based on current conditions to server for training); provide, based on receipt of the training command, the ML model to a training and validation engine to train the ML model (Shen, para [0216, 0482], learning and validation of ML model); and provide a fully trained version of the ML model to the client device after completion of the training of the ML model (Shen, para [0216], transmits trained models over network to clients).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the training to include the steps the Shen. The motivation for doing so would have been to provide data to a training system from remote users (Shen, para [0003]).
Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over Cataltepe United States Patent Application Publication US 2019/0279102 in view of Kolar United States Patent Application Publication US 2021/0304061 in further view of Ostergaard United States Patent Application Publication US 2022/0277231 as modified by Lee United States Patent Application Publication US 2022/0114453.
Regarding claim 26, Cataltepe in view of Kolar in further view of Ostergaard discloses the one or more NTCRM of claim 1. Cataltepe in view of Kolar in further view of Ostergaard does not disclose the additional limitations of the present claim.
Lee discloses wherein execution of the instructions is to cause the computing system to evaluate an accuracy of the corresponding ML model based on at least one of (1) a Jacobian covariance or (2) a channel pruning with parameter loss estimations to project performance (Lee, para [0071], channel pruning).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the system to include channel pruning. The motivation for doing so would have been to reduce complexity of increasing machine learning models (Lee, para [0004]).
Claim 30 is rejected under 35 U.S.C. 103 as being unpatentable over Cataltepe United States Patent Application Publication US 2019/0279102 in view of Kolar United States Patent Application Publication US 2021/0304061 in further view of Ostergaard United States Patent Application Publication US 2022/0277231 in further view of Lee United States Patent Application Publication US 2022/0114453, in further view of Abdelfattah United States Patent Application Publication US 2022/0101089.
Regarding claim 30, Cataltepe in view of Kolar in further view of Ostergaard discloses the one or more NTCRM of claim 1. Cataltepe in view of Kolar in further view of Ostergaard does not disclose the additional limitations of the present claim.
Lee discloses wherein the instructions cause the one or more processors to identify a level of ML model accuracy based on channel pruning with parameter loss estimation (Lee, para [0071], channel pruning).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the system to include identify a second level of ML model accuracy based on channel pruning with parameter loss estimation. The motivation for doing so would have been to reduce complexity of increasing machine learning models (Lee, para [0004]).
Cataltepe in view of Kolar in further view of Ostergaard in further view of Lee does not disclose to identify a first level of ML model accuracy based on a Jacobian covariance and a second level of ML model accuracy.
Abdelfattah discloses to identify a first level of ML model accuracy based on a Jacobian covariance and a second level of ML model accuracy (Abdelfattah, para [0060] teaches identifying multiple levels of model accuracy, with one being based on Jacobian covariance).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the system to include identifying a first level of ML model accuracy based on a Jacobian covariance and a second level of ML model accuracy. The motivation for doing so would have been to achieve better correlation with respect to the final accuracy (Abdelfattah, para [0060]).
Response to Arguments
Applicant's arguments filed 01/14/2026 have been fully considered with regards to the 35 U.S.C. 102/103 rejection, but they are not persuasive.
The applicant asserts on page 10 of the remarks “the learned relationships corresponding to a mapping between the one or more actual performance metrics and the corresponding ML models, wherein the one or more actual performance metrics include accuracy, latency, and power consumption. The alleged Cateltepe/Kolar/Ostergaard combination fails to teach or suggest such instructions.”.
The examiner respectfully disagrees, as the BRI of “the one or more actual performance metrics include accuracy, latency, and power consumption” is one or more performance metrics that comprise any of one accuracy, latency, and power consumption. Para [0116] of Cateltepe discloses learned relationships corresponding to a mapping between the corresponding ML models and the one or more actual performance metrics, such as accuracy and delays (latency).
Claims 11 and 17, comprising amended elements substantially similar to that of claim 1, are rejected on a similar basis.
Claims dependent on independent claims do not overcome the deficiencies of the rejected independent claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUMAIRA ZAHIN MAUNI whose telephone number is (703)756-5654. The examiner can normally be reached Monday - Friday, 9 am - 5 pm (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MATTHEW ELL can be reached at (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.Z.M./Examiner, Art Unit 2141
/ANDREW L TANK/Primary Examiner, Art Unit 2141
Prosecution Insights