Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is in response to an application filed on February 14th, 2023. Claims 1-20 are pending in the current application.
Claim Interpretation
The “one or more tangible processor-readable storage media” of claims 15-20 are defined in the specification to exclude transitory signals per se. (See Paragraph 54)
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1, Under Step 1 of the Subject Matter Eligibility Test of Products and
Processes, the claim is directed towards a method which is considered a process, which is one of the four statutory categories.
Next, under a Step 2A Prong 1 Analysis, the following excerpts from the claim recites a grouping of abstract idea:
Tuning hyperparameters of a machine learning model (mental process)
generating, for each hyperparameter, a performance attribution statistic corresponding to an evaluation metric of the machine learning model based on historical experiment statistics for the evaluation metric and the machine learning model (mental process)
allocating a weight to each hyperparameter based on the performance attribution statistic of the hyperparameter (mental process)
updating, in a series of experiments, the hyperparameters based on the weight assigned to each hyperparameter (mental process)
selecting a set of the hyperparameters for the machine learning model from one of the experiments, wherein the set of the hyperparameters results in a recorded value of the evaluation metric that satisfies a tuning condition. (mental process)
Therefore, we have to examine the claim under Step 2A prong 2, which considers the additional elements within the claim. Since there are no additional elements within the claim, the claim is directed towards an abstract idea.
Under a Step 2B analysis, the claim's additional elements do not amount to significantly
more than the judicial exception as explained above in Step 2A prong 2. Therefore, the claim is ineligible.
Regarding claim 8, Under Step 1 of the Subject Matter Eligibility Test of Products and
Processes, the claim is directed towards a system which is considered a machine, which is one of the four statutory categories.
Next, under a Step 2A Prong 1 Analysis, the following excerpts from the claim recites a grouping of abstract idea:
tuning hyperparameters of a machine learning model (mental process)
generate, for each hyperparameter, a performance attribution statistic corresponding to an evaluation metric of the machine learning model based on historical experiment statistics for the evaluation metric and the machine learning model (mental process)
allocate a weight to each hyperparameter based on the performance attribution statistic of the hyperparameter (mental process)
update, in a series of experiments, the hyperparameters based on the weight assigned to each hyperparameter; (mental process)
select a set of the hyperparameters for the machine learning model from one of the experiments wherein the set of the hyperparameters results in a recorded value of the evaluation metric that satisfies a tuning condition. (mental process)
Therefore, we have to examine the claim under Step 2A prong 2, which considers the additional elements within the claim. The additional elements include:
One or more hardware processors
A performance attributor
A hyperparameter weight assessor
A hyperparameter updater
and a hyperparameter selector
The limitations, as drafted, merely acts as an indication of the technological environment or field of use, and “generally links” the judicial exception, to processors, an attributor, an updater, an assessor, and a selector that can perform it. (See MPEP 2106.05(h)) Therefore, the claim is directed towards an abstract idea.
Under a Step 2B analysis, the claim's additional elements do not amount to significantly
more than the judicial exception as explained above in Step 2A prong 2. Therefore, the claim is ineligible.
Regarding claim 15, Under Step 1 of the Subject Matter Eligibility Test of Products and
Processes, the claim is directed towards a medium which is considered a manufacture, which is one of the four statutory categories.
Next, under a Step 2A Prong 1 Analysis, the following excerpts from the claim recites a grouping of abstract idea:
A process of tuning hyperparameters of a machine learning model (mental process)
generating, for each hyperparameter, a performance attribution statistic corresponding to an evaluation metric of the machine learning model based on historical experiment statistics for the evaluation metric and the machine learning model (mental process)
allocating a weight to each hyperparameter based on the performance attribution statistic of the hyperparameter; (mental process)
updating, in a series of experiments, the hyperparameters based on the weight assigned to each hyperparameter (mental process)
and selecting a set of the hyperparameters for the machine learning model from one of the experiments, wherein the set of the hyperparameters results in a recorded value of the evaluation metric that satisfies a tuning condition. (mental process)
Therefore, we have to examine the claim under Step 2A prong 2, which considers the additional elements within the claim. The additional elements include:
One or more tangible processor-readable storage media embodied with instructions for executing on one or more processors and circuits of a computing device
The limitation, as drafted, merely indicates the technological environment or field of use, and “generally links” the judicial exception, to media, processors and circuitry that can perform it. (See MPEP 2106.05(h)) Therefore, the claim is directed towards an abstract idea.
Under a Step 2B analysis, the claim's additional elements do not amount to significantly
more than the judicial exception as explained above in Step 2A prong 2. Therefore, the claim is ineligible.
Regarding claims 2, 9, and 16, the claims recite “the evaluation metric corresponds to a performance objective of the machine learning model to which the hyperparameters are being tuned.” The limitation, as drafted , is considered to be a “mental process”, which is a grouping of abstract idea. Therefore, the claims are rejected for the same reasons as set forth in the rejections of claims 1, 8, and 15.
Regarding claims 3, 10, and 17, the claims recite “the historical experiment statistics track historical values of the evaluation metric against different values of each hyperparameter.” The limitation, as drafted , is considered to be a “mental process”, which is a grouping of abstract idea. Therefore, the claims are rejected for the same reasons as set forth in the rejections of claims 1, 8, and 15.
Regarding claims 4, 11, and 18, the claims recite “wherein the performance attribution statistic corresponding to the hyperparameter indicates a sensitivity of the evaluation metric to changes in the hyperparameter” The limitation, as drafted , is considered to be a “mental process”, which is a grouping of abstract idea. Therefore, the claims are rejected for the same reasons as set forth in the rejections of claims 1, 8, and 15.
Regarding claim 5, 12, and 19, the claims recite “for multiple iterations limited by a compute budget, the hyperparameter updater is further configured to: randomly select a hyperparameter based on the weight allocated to the hyperparameter,”, “update the hyperparameter to a new value”, “execute an experiment on the machine learning model based on the new value of the hyperparameter,”, and “record a value of the evaluation metric resulting from the experiment.” The limitations, as drafted, all are considered to be “mental processes”, which is a grouping of abstract idea. Therefore, the claims are rejected for the same reasons as set forth in the rejections of claims 1, 8, and 15.
Regarding claims 6, 13, and 20, the claims recite “the hyperparameter updater is further configured to update the hyperparameter to a new value based on maximizing a growth rate of the evaluation metric based on changes in the hyperparameter and minimizing a covariance of the evaluation metric based on changes in the hyperparameter.” The limitation, as drafted, under the broadest reasonable interpretation, is interpreted to be a “mathematical concept”, which is a grouping of abstract idea. Therefore, the claims are rejected for the same reasons as set forth in the rejections of claims 1, 8, and 15.
Regarding claims 7 and 14, the claims recite “the hyperparameter updater is further configured to update the hyperparameter to a new value using a Bayesian model.” The limitation, as drafted, under the broadest reasonable interpretation, is interpreted to be a “mathematical concept”, which is a grouping of abstract idea. Therefore, the claims are rejected for the same reasons as set forth in the rejections of claims 1 and 8.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-5, 8-12, and 15-19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Honghe Jin. (Herein referred to as Jin) (Hyperparameter Importance for Machine Learning Algorithms)
Regarding claim 1, Jin teaches a method of tuning hyperparameters of a machine learning model, the method comprising: generating, for each hyperparameter, a performance attribution statistic corresponding to an evaluation metric of the machine learning model based on historical experiment statistics for the evaluation metric and the machine learning model (“Following the notations in Section 2.1, the importance of the jth hyperparameter θj is defined by Varθj (R∗ (θ)), which is the variance of the risk on the hyperparameter θj .”, pg. 2, under “2.2 Hyperparameter Importance… Definition 1”) (The “variance of risk” acted as a performance attribution statistic, generated for each hyperparameter. It corresponds to variance reduction, which is an evaluation metric, (“In the estimation of the hyperparameter importance, we can sample from the entire data set multiple times to reduce the variance of n(θ).”, pg. 4, first paragraph) and is based on previous experiment data from previous subsamples of data. (See Figure 1)) allocating a weight to each hyperparameter based on the performance attribution statistic of the hyperparameter (“Although the values of variances are different when we integral out other hyperparameters, the essential terms in (1) and (2) are equivalent when we rank the importance of different hyperparameters.”, pg. 3, under the bullets at the top of the page) (The rankings of importance for the hyperparameters are decided based on the variance of risk between the parameters, indicating a weight.) updating, in a series of experiments, the hyperparameters based on the weight assigned to each hyperparameter (“With the estimated hyperparameter importance, we tune the hyperparameters sequentially by group with the importance rankings.”, pg. 6, first paragraph) (The example shows hyperparameter being updated and tuned based on the rankings.) and selecting a set of the hyperparameters for the machine learning model from one of the experiments, wherein the set of the hyperparameters results in a recorded value of the evaluation metric that satisfies a tuning condition. (“The best hyperparameters selected by the two tuning methods are shown in Table 1. The sequential hyperparameter tuning with the estimated importance yields similar selected hyperparameters and AUC.”, pg. 6, paragraph 3; See also Table 1) (The tuning condition is related to the area under the curve, and variance of risk.)
Regarding claim 8, Jin teaches a computing system for tuning hyperparameters of a machine learning model, the computing system comprising: one or more hardware processors (While Jin never explicitly mentions processors, you would implicitly need one or more processors to run Jin’s method.) a performance attributor executable by the one or more hardware processors configured to generate, for each hyperparameter, a performance attribution statistic corresponding to an evaluation metric of the machine learning model based on historical experiment statistics for the evaluation metric and the machine learning model (“Following the notations in Section 2.1, the importance of the jth hyperparameter θj is defined by Varθj (R∗ (θ)), which is the variance of the risk on the hyperparameter θj .”, pg. 2, under “2.2 Hyperparameter Importance… Definition 1”) (The “variance of risk” acted as a performance attribution statistic, generated for each hyperparameter. It corresponds to variance reduction, which is an evaluation metric, (“In the estimation of the hyperparameter importance, we can sample from the entire data set multiple times to reduce the variance of n(θ).”, pg. 4, first paragraph) and is based on previous experiment data from previous subsamples of data. (See Figure 1)) a hyperparameter weight assessor executable by the one or more hardware processors and configured to allocate a weight to each hyperparameter based on the performance attribution statistic of the hyperparameter (“Although the values of variances are different when we integral out other hyperparameters, the essential terms in (1) and (2) are equivalent when we rank the importance of different hyperparameters.”, pg. 3, under the bullets at the top of the page) (The rankings of importance for the hyperparameters are decided based on the variance of risk between the parameters, indicating a weight.) a hyperparameter updater executable by the one or more hardware processors and configured to update, in a series of experiments, the hyperparameters based on the weight assigned to each hyperparameter; (“With the estimated hyperparameter importance, we tune the hyperparameters sequentially by group with the importance rankings.”, pg. 6, first paragraph) (The example shows hyperparameter being updated and tuned based on the rankings.) and a hyperparameter selector executable by the one or more hardware processors and configured to select a set of the hyperparameters for the machine learning model from one of the experiments wherein the set of the hyperparameters results in a recorded value of the evaluation metric that satisfies a tuning condition. (“The best hyperparameters selected by the two tuning methods are shown in Table 1. The sequential hyperparameter tuning with the estimated importance yields similar selected hyperparameters and AUC.”, pg. 6, paragraph 3; See also Table 1) (The tuning condition is related to the area under the curve, and variance of risk.)
Regarding claim 15, Jin teaches one or more tangible processor-readable storage media embodied with instructions for executing on one or more processors and circuits of a computing device a process of tuning hyperparameters of a machine learning model, (While Jin never explicitly mentions one or more tangible processor-readable storage media, you would implicitly need one to be able to distribute the method of Jin.) the process comprising: generating, for each hyperparameter, a performance attribution statistic corresponding to an evaluation metric of the machine learning model based on historical experiment statistics for the evaluation metric and the machine learning model (“Following the notations in Section 2.1, the importance of the jth hyperparameter θj is defined by Varθj (R∗ (θ)), which is the variance of the risk on the hyperparameter θj .”, pg. 2, under “2.2 Hyperparameter Importance… Definition 1”) (The “variance of risk” acted as a performance attribution statistic, generated for each hyperparameter. It corresponds to variance reduction, which is an evaluation metric, (“In the estimation of the hyperparameter importance, we can sample from the entire data set multiple times to reduce the variance of n(θ).”, pg. 4, first paragraph) and is based on previous experiment data from previous subsamples of data. (See Figure 1)) allocating a weight to each hyperparameter based on the performance attribution statistic of the hyperparameter; (“Although the values of variances are different when we integral out other hyperparameters, the essential terms in (1) and (2) are equivalent when we rank the importance of different hyperparameters.”, pg. 3, under the bullets at the top of the page) (The rankings of importance for the hyperparameters are decided based on the variance of risk between the parameters, indicating a weight.) updating, in a series of experiments, the hyperparameters based on the weight assigned to each hyperparameter (“With the estimated hyperparameter importance, we tune the hyperparameters sequentially by group with the importance rankings.”, pg. 6, first paragraph) (The example shows hyperparameter being updated and tuned based on the rankings.) and selecting a set of the hyperparameters for the machine learning model from one of the experiments, wherein the set of the hyperparameters results in a recorded value of the evaluation metric that satisfies a tuning condition. (“The best hyperparameters selected by the two tuning methods are shown in Table 1. The sequential hyperparameter tuning with the estimated importance yields similar selected hyperparameters and AUC.”, pg. 6, paragraph 3; See also Table 1) (The tuning condition is related to the area under the curve, and variance of risk.)
Regarding claims 2, 9, and 16, Jin teaches the method, system, and media of claims 1, 8, and 15 respectively, wherein the evaluation metric corresponds to a performance objective of the machine learning model to which the hyperparameters are being tuned. (“We use the gradient boosting machine (GBM) to predict the contractual default of each customer. The hyperparameters being studied are Max Depth, Step Size, Max Iteration, Subsample Rate, Max Bins, and Min Instance Per Node…we can only tune the most important hyperparameters on the entire data so as to save computational resources. In this case, we only need to tune Max Depth, Step Size, and Max Iteration. We tune Max Iteration here because the combined effect of Step Size & Max Iteration is the second most important. If the computation resources are very limited, tuning only Max Depth and Step Size should give a fairly good result as well.”, pg. 6, second to last paragraph; pg. 7, second paragraph) (In the example, the evaluation metric (variance of risk) helps rank the hyperparameters, which are then chosen by rank to be tuned. This is done to predict the contractual default of each customer while saving computational resources, which acts as our performance objective.)
Regarding claims 3, 10, and 17, Jin teaches the method, system, and media of claims 1, 8, and 15 respectively, wherein the historical experiment statistics track historical values of the evaluation metric against different values of each hyperparameter. (“In addition to the importance of a single hyperparameter, we can also study the effect of hyperparameter combinations. For example, the combination importance of (θj , θk) can be defined as Varθj ,θk (R∗ (θ)), in which we calculate the variance of R∗ (θ) with varying (θj , θk) and integral out other hyperparameters. This can help us study the joint effect of some confounding hyperparameters, such as the number of trees and the step size in the gradient boost models.”, pg. 3, third paragraph, See also Figure 1: Hyperparameter (HP) importance estimation procedures via subsampling.) (Figure 1 show the tracking of variance of risk between different combination of hyperparameters)
Regarding claims 4, 11, and 18, Jin teaches the method, system, and media of claims 1, 8, and 15 respectively, wherein the performance attribution statistic corresponding to the hyperparameter indicates a sensitivity of the evaluation metric to changes in the hyperparameter (“The intuition behind Definition 1 is that Varθj (R∗ (θ)) measures the variability of the risk when θj is changed. If the risks change much when θj varies, then θj should be tuned carefully since it can influence the results.”, pg. 2, second to last paragraph) (Variance of risk intrinsically indicates a sensitivity to changes in the hyperparameter.)
Regarding claims 5, 12, and 19, Jin teaches the method, system, and media of claims 1, 8, and 15 respectively, wherein the updating comprises: for multiple iterations limited by a compute budget, selecting a hyperparameter based on the weight allocated to the hyperparameter, (“Since there is a big jump in the importance trend line in Figure 3, panel (a), we can only tune the most important hyperparameters on the entire data so as to save computational resources…If the computation resources are very limited, tuning only Max Depth and Step Size should give a fairly good result as well.”, pg. 7, second paragraph) updating the hyperparameter to a new value, executing an experiment on the machine learning model based on the new value of the hyperparameter (See Figure 3. The values of the selected hyperparameters are tuned after every iteration.) and recording a value of the evaluation metric resulting from the experiment (As shown in Figure 1, there is a functionality to record the value the variance of risk between hyperparameters, resulting from experiments)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 6, 13, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Jin in view of Bobak Shahriari et al. (Herein referred to as Shahriari) (Taking the Human Out of the Loop: A Review of Bayesian Optimization)
Regarding claims 6, 13, and 20, Jin teaches the method, system, and media of claims 1, 8, and 15 respectively. However, Jin does not explicitly teach updating the hyperparameter to a new value based on maximizing a growth rate of the evaluation metric based on changes in the hyperparameter and minimizing a covariance of the evaluation metric based on changes in the hyperparameter.
Shahriari teaches updating the hyperparameter to a new value based on maximizing a growth rate of the evaluation metric based on changes in the hyperparameter (“These acquisition functions trade off exploration and exploitation; their optima are located where the uncertainty in the surrogate model is large (exploration) and/or where the model prediction is high (exploitation). Bayesian optimization algorithms then select the next query point by maximizing such acquisition functions”, pg. 150; See also Fig. 1) (Fig. 1 shows graphic changes based on exploitation and exploration, indicating some sort of rate) and minimizing a covariance of the evaluation metric based on changes in the hyperparameter. (“One early approach to modeling large n with GPs considered using m G n inducing pseudoinputs to reduce the rank of the covariance matrix to m, resulting in a significant reduction in computational cost.” pg. 158, left column, top of the page)
Therefore, it would have been considered obvious to one of ordinary skill in the art,
prior to the current application’s filing date, to combine the hyperparameter selection and tuning of Jin with the exploration, exploitation, and covariance reduction of Shahriari. One would be motivated to combine the two teachings, prior to the filing date of the current application, as these functionality allow for cheaper computational resources, as disclosed in Shahriari. (“Naturally, these acquisition functions are often even more multimodal and difficult to optimize, in terms of querying efficiency, than the original black-box function f. Therefore, it is critical that the acquisition functions be cheap to evaluate or approximate: cheap in relation to the expense of evaluating the black box f… One early approach to modeling large n with GPs considered using m G n inducing pseudoinputs to reduce the rank of the covariance matrix to m, resulting in a significant reduction in computational cost.” pg. 150, right column; pg. 158, left column, top of the page)
Claims 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Jin in view of Baohe Zhang et al. (Herein referred to as Zhang) (On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learning)
Regarding 7 and 14, Jin teaches the method, system, and media of claims 1, 8, and 15 respectively. However, Jin does not explicitly teach updating the hyperparameter to a new value using a Bayesian model.
Zhang teaches updating the hyperparameter to a new value using a Bayesian model. (“BO sequentially optimizes an algorithm’s hyperparameters by using previous evaluation results to select the next set of hyperparameters to evaluate. The previous evaluations are used to build a surrogate which models the relationship between hyperparameter values and resulting performance. Then, an acquisition function is used to trade off exploration and exploitation.”, pg. 2, left column, under “2 Related Work”)
Therefore, it would have been considered obvious to one of ordinary skill in the art, prior to the current application’s filing date, to combine the hyperparameter selection and tuning of Jin with the Bayesian model of Zhang. One would be motivated to combine the two teachings, prior to the filing date of the current application, as Bayesian Optimization (BO) optimizes hyperparameters based on previous evaluation results, as disclosed in Zhang. (“BO sequentially optimizes an algorithm’s hyperparameters by using previous evaluation results to select the next set of hyperparameters to evaluate. The previous evaluations are used to build a surrogate which models the relationship between hyperparameter values and resulting performance. Then, an acquisition function is used to trade off exploration and exploitation. The sequential nature of BO and use of full function evaluations however can make it very costly”, pg. 2, left column, under “2 Related Work”)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Tyler E Iles whose telephone number is (571)272-5442. The examiner can normally be reached 9:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TYLER EDWARD ILES/Patent Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights