Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This Final Office action is in reply to the amendments/remarks filed 8/18/2025.
Claims 1, 3-5, 7, 9-11 and 16 have been amended.
Claims 8 and 20 have been cancelled.
Claims 2, 12 and 17 were previously cancelled.
Claim 21 is a new claim.
Claims 1, 3-7, 9-11, 13-16, 18, 19 and 21 are pending.
Response to Amendments/Arguments
Applicant’s amendments to independent claims 1, 11 and 16 overcome the 35USC112 rejections, they have been withdrawn.
With respect to applicant’s arguments regarding the 35 USC 103 rejection, applicant argues the amended limitations, however applicant’s amendments necessitated new grounds of rejection. Therefore, applicant’ arguments are moot. Examiner has modified the rejection to further explain how the claims are being interpreted and addressed each of applicant’s claim limitations as noted below in this Final action.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3-7, 9-11, 13-16, 18, 19 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Kishimoto et al., US Patent Application Publication No US 2023/0054582 A1, in view of Song et al., Foreign Patent CN104809522A.
With respect to claims 1, 11 and 16,
Kishimoto discloses,
receiving, by a carbon tuner engine, a carbon budget constraint, for carbon emission bounded training a machine learning model; (¶5: “a system for simultaneous feature selection and hyperparameter optimization of non-linear models of machine learning is provided. The system includes a memory and one or more processors in communication with the memory configured to set a first solution having first hyperparameters and a first set of features selected from a plurality of features of a training data set”; Fig. 1, ¶28: “The training data 10 is supplied for data transformation and model evaluation. Data transformation involves feature selection (FS) 12. Model evaluation involves training and testing the model via train and test component 14. The data transformation and model evaluation are implemented with the systematic search based on LDS with a weighted table 20”; ¶29: “Both feature selection 12 and hyperparameter tuning 16 are key tasks in machine learning”; ¶31: “FIG. 2 illustrates a practical application for the machine learning workflow for computational material discovery, in accordance with an embodiment of the present invention”; Fig 4, ¶48: “provide a training data set including data related to a plurality of features”)
sampling, by a scheduling module, a search space of the machine learning model (Fig 1, ¶5: “The system includes a memory and one or more processors in communication with the memory configured to set a first solution having first hyperparameters and a first set of features selected from a plurality of features of a training data set, initialize a weight table providing a score for each feature of the first set of features, initialize a discrepancy, perform a limited discrepancy search (LDS), according to an order based on the weight table, to obtain a second solution having second hyperparameters and a second set of features from the plurality of features by swapping the first set of features and switching the first hyperparameters from the first solution with the discrepancy, while updating the weight table during LDS, compare the second solution with the first solution, and obtain a new solution with improved features and improved hyperparameters, as an optimized solution”)
identifying, within the search space, a set of hyperparameters of the machine learning model ¶29: “Both hyperparameter tuning 16 and feature selection 12 are often useful to increase model performance. Feature selection 12 is also undertaken to attain sparse models. Sparsity may yield better model interpretability and lower cost of data acquisition, data handling, and model inference. While sparsity may have a beneficial or detrimental effect on predictive performance, a small drop in performance may be acceptable in return for a substantial gain in sparseness. Feature selection 12 can therefore be treated as a multi-objective optimization task. As a result, the exemplary embodiments perform hyperparameter tuning 16 and feature selection 12 simultaneously or concurrently because the choice of features of a model may influence what hyperparameters perform well. After employment of the systematic search based on LDS with a weighted table 20, an optimized model 22 is obtained. The optimized model 22 is then deployed via a model deployment component 24”; Fig 3, ¶38: “At block 50, receive an initial feature set and hyperparameters (HPs) (denoted as I) as input, as well as the max discrepancy maxd”; ¶45: “At block 64, determine if d≥maxd. If YES, proceed to block 66 to return I as an optimized solution. If NO, proceed to block 68.”; Fig 4, ¶48: “provide a training data set including data related to a plurality of features”; ¶53: “At block 80, compare the second solution with the first solution. If the second solution is better or superior than the first solution, updating the first solution with the second solution and recurring the LDS up to given max discrepancy. If the second solution is not better or superior than the first solution, incrementing the discrepancy and recurring the LDS up to the given max discrepancy”; ¶54: “At block 82, obtain the first solution finally obtained, as an optimized solution”)
the Bayesian probability based on prior training runs of similar machine learning models (¶23: “Once the new result is obtained, the process of refitting the surrogate function and acquisition is repeated for a fixed number of iterations. One notable case of such optimization procedure is Bayesian Optimization that uses Gaussian Process (GP) for estimating the error dependence on the hyperparameters. The main advantage of this optimization is that it can naturally cope with the stochastic nature of the training/test error of various ML algorithms, as it happens when training neural networks”)
generating, by the carbon tuner engine a training plan for the machine learning model, the training plan operating within the carbon budget constraint (Abstract, ¶5: “a system for simultaneous feature selection and hyperparameter optimization of non-linear models of machine learning is provided. The system includes a memory and one or more processors in communication with the memory configured to set a first solution having first hyperparameters and a first set of features selected from a plurality of features of a training data set, initialize a weight table providing a score for each feature of the first set of features, initialize a discrepancy, perform a limited discrepancy search (LDS), according to an order based on the weight table, to obtain a second solution having second hyperparameters and a second set of features from the plurality of features by swapping the first set of features and switching the first hyperparameters from the first solution with the discrepancy, while updating the weight table during LDS, compare the second solution with the first solution, and obtain a new solution with improved features and improved hyperparameters, as an optimized solution”; Fig 1, ¶27: “FIG. 1 shows an exemplary machine learning workflow employing Limited Discrepancy Search (LDS) with a weight table, in accordance with an embodiment of the present invention; ¶28: “training data 10 is supplied for data transformation and model evaluation. Data transformation involves feature selection (FS) 12. Model evaluation involves training and testing the model via train and test component 14. The data transformation and model evaluation are implemented with the systematic search based on LDS with a weighted table 20.; ¶29: “Both hyperparameter tuning 16 and feature selection 12 are often useful to increase model performance…Feature selection 12 is also undertaken to attain sparse models. Sparsity may yield better model interpretability and lower cost of data acquisition, data handling, and model inference. While sparsity may have a beneficial or detrimental effect on predictive performance, a small drop in performance may be acceptable in return for a substantial gain in sparseness. Feature selection 12 can therefore be treated as a multi-objective optimization task. As a result, the exemplary embodiments perform hyperparameter tuning 16 and feature selection 12 simultaneously or concurrently because the choice of features of a model may influence what hyperparameters perform well. After employment of the systematic search based on LDS with a weighted table 20, an optimized model 22 is obtained. The optimized model 22 is then deployed via a model deployment component 24”)
monitoring, by the carbon tuner engine, carbon emissions while training the machine learning model according to the training plan (¶5: “while updating the weight table during LDS, compare the second solution with the first solution, and obtain a new solution with improved features and improved hyperparameters, as an optimized solution”; Fig 1, ¶27: “FIG. 1 shows an exemplary machine learning workflow employing Limited Discrepancy Search (LDS) with a weight table, in accordance with an embodiment of the present invention; ¶28: “training data 10 is supplied for data transformation and model evaluation. Data transformation involves feature selection (FS) 12. Model evaluation involves training and testing the model via train and test component 14. The data transformation and model evaluation are implemented with the systematic search based on LDS with a weighted table 20”; ¶31: “FIG. 2 illustrates a practical application for the machine learning workflow for computational material discovery, in accordance with an embodiment of the present invention; Fig 3, ¶42: “At block 58, perform LDS with d and I to find a better or superior feature set and HPs while updating the values in the weight table”)
tuning, by the carbon tuner engine, the set of hyperparameters to cause the machine learning model to operate within a pre-defined carbon emission performance criterion; updating, by the carbon tuner engine, the training plan of the machine learning model based on the tuned set of hyperparameters (¶2: Hyperparameters have to be tuned in order to obtain an optimized model given a performance or cost function. Both of these techniques, that is, feature selection and hyperparameter optimization, have a huge impact on model quality, interpretability, training speed, and model evaluation speed where an optimized solution comes as a compromise between these model properties”; ¶5: “updating the weight table during LDS, compare the second solution with the first solution, and obtain a new solution with improved features and improved hyperparameters, as an optimized solution”; ¶28: “The training data 10 is supplied for data transformation and model evaluation. Data transformation involves feature selection (FS) 12. Model evaluation involves training and testing the model via train and test component 14. The data transformation and model evaluation are implemented with the systematic search based on LDS with a weighted table 20”; ¶29: “Both feature selection 12 and hyperparameter tuning 16 are key tasks in machine learning. Both hyperparameter tuning 16 and feature selection 12 are often useful to increase model performance… the exemplary embodiments perform hyperparameter tuning 16 and feature selection 12 simultaneously or concurrently because the choice of features of a model may influence what hyperparameters perform well. After employment of the systematic search based on LDS with a weighted table 20, an optimized model 22 is obtained. The optimized model 22 is then deployed via a model deployment component 24”; Fig 3, Fig 4, ¶37-¶46; ¶44: “determine if d<maxd and no better or superior solution is found. If YES, proceed to block 62, where d is incremented. If NO, proceed to block 64”; Fig 3, ¶45: “At block 64, determine if d≥maxd. If YES, proceed to block 66 to return I as an optimized solution. If NO, proceed to block 68”; ¶45: “determine if d≥maxd. If YES, proceed to block 66 to return I as an optimized solution. If NO, proceed to block 68”; ¶46: “At block 68, reset I with the solution found by LDS”; ¶47: “a method for simultaneous feature selection (FS) and hyperparameter optimization (HPO) of non-linear models”; Fig 4, ¶53: “At block 80, compare the second solution with the first solution. If the second solution is better or superior than the first solution, updating the first solution with the second solution and recurring the LDS up to given max discrepancy. If the second solution is not better or superior than the first solution, incrementing the discrepancy and recurring the LDS up to the given max discrepancy”;
one or more computer processors (¶5: “a memory and one or more processors in communication with the memory; one or more computer readable storage media; and computer program instructions to (¶4: “The computer program product includes a computer readable storage medium having program instructions embodied therewith,”)
A computer program product for training a machine learning model within a carbon budget constraint, the computer program product comprising one or more computer readable storage media and program instructions sorted on the one or more computer readable storage media to (¶4: “a computer program product for simultaneous feature selection and hyperparameter optimization of non-linear models of machine learning is provided. The computer program product includes a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer”; claim 8: “the program instructions executable by a computer to cause the computer to: set a first solution having first hyperparameters and a first set of features selected from a plurality of features of a training data set”)
Kishimoto discloses all of the above limitations, Kishimoto does not distinctly describe carbon emissions; carbon budget constraints, but Song however as shown discloses,
carbon emissions; carbon budget constraints; the carbon budget constraint indicating a carbon emission limit for performing the training (¶88: “The present invention is a linear mapping comprehensive energy forecasting method. In the energy consumption demand forecasting calculation of a large amount of clean energy connected to the power grid under multiple environments such as energy economy environment industrial structure adjustment, multiple factors such as industrial structure, carbon emission constraints, energy prices, etc. are analyzed, selected and added and quantified, such as average electricity price, actual output of primary energy of electricity, international crude oil price, domestic gasoline price, domestic diesel price, domestic fuel oil price, coal price, etc., and self-optimizing variable weight calculation is performed on various factors affecting energy consumption changes to ensure objective reflection of the relationship between various factors and forecast results, establish a high-dimensional feature space, and extract and combine and optimize the most discriminative low-dimensional features from it, fully train data, maximize the avoidance of over-learning, improve algorithm efficiency, make the energy consumption forecasting process faster and more effective, and help improve the accuracy of energy forecasting under the new normal of economic development and strong constraints of energy environment, and then calculate regional energy balance, and finally determine reasonable and feasible energy development and security policies”; Finally, the classifiers in all iterations are integrated and the regression equation is obtained by weighting. That is, for all weak classifiers in all iterative layers determined in step 3.5, the voting parameters determined in step 3.7 are used as corresponding weights, and all classifiers are weighted integrated to obtain the regression equation. The regression equation describes the implicit function relationship between sample data and target energy consumption demand value. Its input is 10 basic data of the target year, and its output is the predicted value of regional energy consumption demand data “; ¶204: “Although the value of the regression function is the result obtained by the classifier operation, however the y0 and x0 of the above formula must satisfy the equality constraint H (x0, y0) = C regression equation on the set D of, for example, if desired to predict energy consumption demand value of 2013 years can be regarded as parameters to construct the new containing undetermined parameter sample set (x0, y0) and participate in building the regression equation, and the final classification value equal to C”)
Kishimoto teaches a method/system for feature dataset selection and hyperparameter optimization of non-linear models of machine learning. Kishimoto discloses a machine learning workflow employing Limited Discrepancy Search with a weight table whereby training data is supplied, feature selection and hyperparameter tuning, and model evaluation to obtain and deploy an optimized model. Song discloses a comprehensive energy forecasting method/system in energy-economic environments utilizing several factors including but not limiting to carbon emission constraints. Kishimoto and Song are directed to the same field of endeavor since they are related to obtaining and training features/data constraints to improve algorithm efficiencies in a computing environment. A person of ordinary skill in the art would have been motivated to combine the known machine learning techniques/model for feature selection and hyperparameter optimization of Kishimoto with the techniques for energy forecasting (including carbon emission constraints) as taught by Song to achieve the claimed invention (training plan of the machine learning model based on the monitored carbon emissions) with a reasonable expectation of success in doing so (" DyStar Textilfarben GmbH & Co. Deutschland KG v. C.H. Patrick Co., 464 F.3d 1356, 1360, 80 USPQ2d 1641, 1645 (Fed. Cir. 2006)); and the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate such machine learning techniques/models into similar systems, hence improving the accuracy of energy forecasting. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the known machine learning techniques/model for feature selection and hyperparameter optimization of Kishimoto with the techniques for energy forecasting as taught by Song since it allows for optimizing features from fully trained data hence improving algorithm efficiency and making forecasting process faster and more effective (¶88).
With respect to claims 3, 13 and 18,
Kishimoto and Song disclose all of the above limitations, Kishimoto further discloses,
predicting the carbon emissions of a round of training the identified hyperparameters (Fig 2, 3, ¶36: “machine learning provides a new means of screening novel materials with good performance, developing quantitative structure-activity relationships (QSARs) and other models, predicting the properties of materials, discovering new materials and performing other materials-related studies. One exemplary method of machine learning involves systematic local search with LDS and a weight table, as described herein, for computation material discovery (CMD)”; Fig 4, ¶48: “provide a training data set including data related to a plurality of features”; ¶53: “At block 80, compare the second solution with the first solution. If the second solution is better or superior than the first solution, updating the first solution with the second solution and recurring the LDS up to given max discrepancy. If the second solution is not better or superior than the first solution, incrementing the discrepancy and recurring the LDS up to the given max discrepancy”; ¶54: “At block 82, obtain the first solution finally obtained, as an optimized solution”)Fig 6, ¶63: “molecular structures 150 having chemical properties can be used as a dataset to generate a property prediction model 154 for receiving optimized features and hyperparameters. This can be accomplished by employing the systematic search 152 having the LDS with weighted table. Molecular designs 156 with target properties can enable the generation of new molecular structures 158”; ¶64: “One example is shown, where artificial intelligence is applied to a molecular structure 160 to generate predicted properties 164. The artificial intelligence applied can be the exemplary LDS with weighted table 162”; ¶66: “The basic idea of using machine learning methods for material property prediction is to analyze and map the relationships (nonlinear in most cases) between the properties of a material and their related factors by extracting knowledge from existing empirical data. FIGS. 1-6 show the fundamental framework for the application of machine learning in material property prediction by employing an LDS with a weighted table”)
Song further discloses,
carbon emissions; carbon budget constraints (¶88: “The present invention is a linear mapping comprehensive energy forecasting method. In the energy consumption demand forecasting calculation of a large amount of clean energy connected to the power grid under multiple environments such as energy economy environment industrial structure adjustment, multiple factors such as industrial structure, carbon emission constraints, energy prices, etc. are analyzed, selected and added and quantified, such as average electricity price, actual output of primary energy of electricity, international crude oil price, domestic gasoline price, domestic diesel price, domestic fuel oil price, coal price, etc., and self-optimizing variable weight calculation is performed on various factors affecting energy consumption changes to ensure objective reflection of the relationship between various factors and forecast results, establish a high-dimensional feature space, and extract and combine and optimize the most discriminative low-dimensional features from it, fully train data, maximize the avoidance of over-learning, improve algorithm efficiency, make the energy consumption forecasting process faster and more effective, and help improve the accuracy of energy forecasting under the new normal of economic development and strong constraints of energy environment, and then calculate regional energy balance, and finally determine reasonable and feasible energy development and security policies.
Kishimoto teaches a method/system for feature dataset selection and hyperparameter optimization of non-linear models of machine learning. Kishimoto discloses a machine learning workflow employing Limited Discrepancy Search with a weight table whereby training data is supplied, feature selection and hyperparameter tuning, and model evaluation to obtain and deploy an optimized model. Kishimoto and Song are directed to the same field of endeavor since they are related to obtaining and training features/data constraints to improve algorithm efficiencies in a computing environment. A person of ordinary skill in the art would have been motivated to combine the known machine learning techniques/model for feature selection and hyperparameter optimization as taught by Kishimoto with the techniques for energy forecasting (including carbon emission constraints) of Song to achieve the claimed invention (predicting, by the processor, the carbon emissions of a round of training the identified hyperparameters) with a reasonable expectation of success in doing so (" DyStar Textilfarben GmbH & Co. Deutschland KG v. C.H. Patrick Co., 464 F.3d 1356, 1360, 80 USPQ2d 1641, 1645 (Fed. Cir. 2006)); and the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate such machine learning techniques/models into similar systems, hence improving the accuracy of energy forecasting. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the known machine learning techniques/model for feature selection and hyperparameter optimization of Kishimoto with the techniques for energy forecasting as taught by Song since it allows for optimizing features from fully trained data hence improving algorithm efficiency and making forecasting process faster and more effective (¶88).
With respect to claims 4, 14 and 19,
Kishimoto and Song disclose all of the above limitations, Kishimoto further discloses,
wherein receiving the carbon budget constraints further comprises: receiving an architecture type for the machine learning model (¶3: “a computer-implemented method for simultaneous feature selection and hyperparameter optimization of non-linear models of machine learning is provided”; ¶12: “FIG. 4 illustrates a method for simultaneous feature selection (FS) and hyperparameter optimization (HPO) of non-linear models, in accordance with an embodiment of the present invention; ¶18: “FIG. 10 is a schematic diagram of exemplary abstraction model layers, in accordance with an embodiment of the present invention”; Fig 1, ¶28: “Model evaluation involves training and testing the model via train and test component 14”; ¶29: “After employment of the systematic search based on LDS with a weighted table 20, an optimized model 22 is obtained. The optimized model 22 is then deployed via a model deployment component 24”)
With respect to claims 5, 15 and 20,
Kishimoto and Song disclose all of the above limitations, Kishimoto further discloses,
wherein receiving the carbon budget further comprises: customizing a single portion of the machine learning model architecture of the machine learning model. (Fig 1, ¶29: “the exemplary embodiments perform hyperparameter tuning 16 and feature selection 12 simultaneously or concurrently because the choice of features of a model may influence what hyperparameters perform well. After employment of the systematic search based on LDS with a weighted table 20, an optimized model 22 is obtained. The optimized model 22 is then deployed via a model deployment component 24”; Fig 5, ¶61: “Reinforcement learning 130 can include temporal differences 140 and deep adversarial networks 142. Reinforcement learning 130 can further include heuristic methods 132, which include the limited discrepancy search (LDS) with weighted table 134 of the exemplary embodiments of the present invention. The LDS with weighted table 134 is employed for feature selection and hyperparameter optimization 136”)
With respect to claim 6,
Kishimoto and Song disclose all of the above limitations, Kishimoto further discloses,
wherein, one or more specific portions of the single architecture are optimized to operate within a carbon emissions budget, via a model fine tuning customization mode (¶28: “The training data 10 is supplied for data transformation and model evaluation. Data transformation involves feature selection (FS) 12. Model evaluation involves training and testing the model via train and test component 14. The data transformation and model evaluation are implemented with the systematic search based on LDS with a weighted table 20”; ¶29: “Both feature selection 12 and hyperparameter tuning 16 are key tasks in machine learning. Both hyperparameter tuning 16 and feature selection 12 are often useful to increase model performance… the exemplary embodiments perform hyperparameter tuning 16 and feature selection 12 simultaneously or concurrently because the choice of features of a model may influence what hyperparameters perform well. After employment of the systematic search based on LDS with a weighted table 20, an optimized model 22 is obtained. The optimized model 22 is then deployed via a model deployment component 24”; ¶73: “LDS can receive any feature set and HPs calculated by any other approach (e.g., greedy algorithm) as an initial solution and tune them further”)
With respect to claim 7,
Kishimoto and Song disclose all of the above limitations, Kishimoto further discloses,
wherein updating the training plan further comprises: stopping the training of one or more hyperparameters in response to the carbon emissions of the training plan exceeding a predetermined threshold (Fig 3, ¶37-¶46; ¶44: “determine if d<maxd and no better or superior solution is found. If YES, proceed to block 62, where d is incremented. If NO, proceed to block 64”; ¶45: “determine if d≥maxd. If YES, proceed to block 66 to return I as an optimized solution. If NO, proceed to block 68”)
With respect to claim 9,
Kishimoto and Song disclose all of the above limitations, Kishimoto further discloses,
wherein updating the training plan further comprises: assigning training of one or more hyperparameters of the machine learning model to a region that utilizes a percentage of one or more renewable power resources above a threshold. (Fig 1, ¶25: “finding new materials by implementing a Limited Discrepancy Search (LDS) with a weighted table. Such configuration can be beneficial in computational material discovery for generating new molecular structures satisfying target property values”; ¶27: “exemplary machine learning workflow employing Limited Discrepancy Search (LDS) with a weight table ¶28: “The training data 10 is supplied for data transformation and model evaluation. Data transformation involves feature selection (FS) 12. Model evaluation involves training and testing the model via train and test component 14. The data transformation and model evaluation are implemented with the systematic search based on LDS with a weighted table 20”; ¶29: “the exemplary embodiments perform hyperparameter tuning 16 and feature selection 12 simultaneously or concurrently because the choice of features of a model may influence what hyperparameters perform well. After employment of the systematic search based on LDS with a weighted table 20, an optimized model 22 is obtained”; ¶30: “Regarding LDS, many problems of practical interest can be solved using tree search methods because carefully tuned successor ordering heuristics guide the search toward regions of the space that are likely to contain solutions… limited discrepancy search is a backtracking algorithm that searches the nodes of the tree in increasing order of such discrepancies. Such LDS algorithm is employed with (or in combination with) a weight table”; Fig 5, ¶34: “Machine learning algorithms aim to optimize the performance of a certain task by using examples and/or past experience”; ¶35: “FIG. 2, molecule structures and chemical property values 30 are used as a dataset. Chemical features 32 are extracted therefrom. The extracted chemical features are processed by a chemical optimization procedure which includes chemical feature selection (substructure of chemical molecules) 34 and hyperparameter optimization 36, which are enabled by a systematic local search including LDS with a weighted table 40. The molecule optimization procedure results in the generation of new molecular structures 42”; Fig 7-Fig 9, ¶84: The artificial intelligence (AI) accelerator chip 222 can implement the FS and HPO using systematic local search 301, and can be used in a wide variety of practical applications, including, but not limited to, robotics 310, industrial applications 312, mobile or Internet-of-Things (IoT) 314, personal computing 316, consumer electronics 318, server data centers 320, physics and chemistry applications 322, healthcare applications 324, and financial applications 326”; “¶87: “Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources”; ¶91: “Resource pooling: the provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to demand. There is a sense of location independence in that the consumer generally has no control or knowledge over the exact location of the provided resources but can be able to specify location at a higher level of abstraction (e.g., country, state, or datacenter)”; ¶104: “Referring now to FIG. 9, …the types of computing devices 454A-N shown in FIG. 9 are intended to be illustrative only and that computing nodes 410 and cloud computing environment 450 can communicate with any type of computerized device over any type of network and/or network addressable connection (e.g., using a web browser)”; Fig 10, ¶105: “FIG. 10 is a schematic diagram of exemplary abstraction model layers”; ¶108: “management layer 580… Resource provisioning 581 provides dynamic procurement of computing resources and other resources that are utilized to perform tasks within the cloud computing environment. Metering and Pricing 582 provide cost tracking as resources are utilized within the cloud computing environment, and billing or invoicing for consumption of these resources. In one example, these resources can include application software licenses. Security provides identity verification for cloud consumers and tasks, as well as protection for data and other resources. User portal 583 provides access to the cloud computing environment for consumers and system administrators. Service level management 584 provides cloud computing resource allocation and management such that required service levels are met. Service Level Agreement (SLA) planning and fulfillment 585 provide pre-arrangement for, and procurement of, cloud computing resources for which a future requirement is anticipated in accordance with an SLA; ¶109: “Workloads layer 590 provides examples of functionality for which the cloud computing environment can be utilized. Examples of workloads and functions which can be provided from this layer include: mapping and navigation 541; software development and lifecycle management 592; virtual classroom education delivery 593; data analytics processing 594; transaction processing 595; and FS and HPO using systematic local search 301”) Applicant’s disclosure only generically discusses at ¶21: “scheduling module 202 can receive a budget of a zero-carbon footprint. Scheduling module 202 can arrange the training of the machine learning model in a region that relies solely on renewable resources such as wind, solar, or hydroelectric power”. Giving the broadest reasonable interpretation of applicant’s claim limitation in light of the specification, Examiner interprets the machine learning algorithms/techniques/models for optimizing the performance of certain tasks utilizing past experience, chemical feature set, hyperparameters, and max discrepancy as taught by Kishimoto as teaching applicant’s limitation, “training of one or more hyperparameters of the machine learning model to a region that utilizes a percentage of one or more renewable power resources above a threshold”. Further, Kishimoto teaches a method/system for feature dataset selection and hyperparameter optimization of non-linear models of machine learning. Kishimoto discloses a machine learning workflow employing Limited Discrepancy Search with a weight table whereby training data is supplied, feature selection and hyperparameter tuning, and model evaluation to obtain and deploy an optimized model. A person of ordinary skill in the art would have been motivated to modify the known machine learning algorithm/techniques/model for feature dataset selection and hyperparameter optimization as taught by Kishimoto to achieve the claimed invention (training of one or more hyperparameters of the machine learning model to a region that utilizes a percentage of one or more renewable power resources above a threshold”) with a reasonable expectation of success in doing so (" DyStar Textilfarben GmbH & Co. Deutschland KG v. C.H. Patrick Co., 464 F.3d 1356, 1360, 80 USPQ2d 1641, 1645 (Fed. Cir. 2006)); and the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate such machine learning techniques/models into similar systems, hence providing a new solution with improved features and improved hyperparameters, as an optimized solution (Figs 1-10, claim 1, ¶24-¶30, ¶34, ¶35, ¶38, ¶45, ¶48, ¶84, ¶87, ¶105)
With respect to claim 10,
Kishimoto and Song disclose all of the above limitations, Kishimoto further discloses,
wherein generating the training plan further comprises: generating, by the scheduling model, the Bayesian probability for the set of hyperparameters (¶23: “Machine learning algorithm performances further depend on the choice of their hyperparameters that refer to learning rate, regularization constants, nonlinearity type, etc… A more elegant solution is using surrogate optimization algorithms to iteratively map the error dependence on the hyperparameters given the data set. Based on this estimation of the error curve, an acquisition function is used to asses which point in this space is most promising to be evaluated next. Once the new result is obtained, the process of refitting the surrogate function and acquisition is repeated for a fixed number of iterations. One notable case of such optimization procedure is Bayesian Optimization that uses Gaussian Process (GP) for estimating the error dependence on the hyperparameters”)
With respect to claim 21,
Kishimoto and Song disclose all of the above limitations, Kishimoto further discloses,
wherein the Bayesian probability is the probability that tuning a particular hyperparameter will have a greater effect on accuracy of the machine learning model compared to tuning other hyperparameters within the search space (¶23: “Once the new result is obtained, the process of refitting the surrogate function and acquisition is repeated for a fixed number of iterations. One notable case of such optimization procedure is Bayesian Optimization that uses Gaussian Process (GP) for estimating the error dependence on the hyperparameters. The main advantage of this optimization is that it can naturally cope with the stochastic nature of the training/test error of various ML algorithms, as it happens when training neural networks”)
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Zappella et al., US Patent No. US 12,165,082 B1, “Hyperparameter Optimization with Operational Constraints”, relating to hyperparameters for tuning a machine learning system optimized using Bayesian optimization with constraints. The hyperparameter optimization may be performed for a received training set and received constraints.
Kojo et al., US Patent Application Publication No US 20230135611 A1, “Online Platform for Carbon Offsetting with Automatically Identified Impacting Measures”, relating to using machine learning to modify hyperparameter optimization, selecting and allocating Climate Impact Measures (CIM) records using a carbon market engine.
Wu et al., US Patent Application Publication No US 2022/0366318 A1, “Machine Learning Hyperparameter Tuning”, relating to obtaining training data for training the machine learning model and determining a set of hyperparameter permutations of the one or more hyperparameters. For each respective hyperparameter permutation in the set of hyperparameter permutations, the operations include training a unique machine learning model using the training data and the respective hyperparameter permutation and determining a performance of the trained model.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry of a general nature or relating to the status of this application or concerning this communication or earlier communications from the Examiner should be directed to Kimberly L. Evans whose telephone number is 571.270.3929. The Examiner can normally be reached on Monday-Friday, 9:30am-5:00pm. If attempts to reach the examiner by telephone are unsuccessful, the Examiner’s supervisor, Lynda Jasmin can be reached at 571.272.6782.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://portal.uspto.gov/external/portal/pair <http://pair-direct.uspto.gov >. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866.217.9197 (toll-free). Any response to this action should be mailed to: Commissioner of Patents and Trademarks, P.O. Box 1450, Alexandria, VA 22313-1450 or faxed to 571-273-8300. Hand delivered responses should be brought to the United States Patent and Trademark Office Customer Service Window: Randolph Building 401 Dulany Street, Alexandria, VA 22314.
/KIMBERLY L EVANS/Examiner, Art Unit 3629
/LYNDA JASMIN/Supervisory Patent Examiner, Art Unit 3629
Prosecution Insights