DETAILED ACTION
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. Applicant’s submission filed 06 January 2026 [hereinafter Response] has been entered, where:
Claims 1, 5, 7, 9, and 10 have been amended.
Claims 1-10 are pending.
Claims 1-10 are rejected.
Foreign priority is claimed to DE 10 2020 20 671.0, filed 10 July 2020. A certified copy of this paper has been filed 04 October 2021. Accordingly, receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
3. An information disclosure statement was submitted on 06 January 2026 and 15 December 2025. The submission complies with the provisions of 37 CFR 1.97. Accordingly, the Examiner considered the information disclosure statement.
Claim Rejections - 35 U.S.C. § 112
4. The rejection to claims 1-10 under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention is WITHDRAWN in view of the Applicant’s amendments to the claims.
Claim Rejections - 35 U.S.C. § 101
5. 35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
6. Claims 1-10 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 recites a method, which is a process, and thus one of the statutory categories of patentable subject matter. (35 U.S.C. § 101).
However, under Step 2A Prong One, the claim recites the limitations of “[(b)]1 determining an optimal parameterization of the hyperparameters using BOHB (Bayesian optimization (BO) and Hyperband (HB)) for each single training data set of a plurality of different training data sets for computer vision,” “[(c)] assessing all of the optimal parameterizations on all training data sets of the plurality of different training data sets using a normalized metric,” “[(d)] creating a matrix, the matrix including the normalized metric for each of the optimal parameterizations and for each of the training data sets,” “[(e)] determining meta-features for each of the training data sets,” “[(g)] optimizing the decision tree as a function of the meta-features and of the matrix.” These limitations of “[(b),(e))] determining,” “[(c)] assessing,” “[(d)] creating a matrix,” and “[(g)] optimizing” can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinion, and accordingly, are a mental process, (MPEP § 2106.05(f)(2) sub III), which are one of the groupings of abstract ideas. Also, the limitations of “[(b)] determining an optimal parameterization of the hyperparameters using BOHB (Bayesian optimization (BO) and Hyperband (HB)) for each single training data set of a plurality of different training data sets for computer vision,” and “[(d)] creating a matrix, the matrix including the normalized metric for each of the optimal parameterizations and for each of the training data sets,” and “[(g)] optimizing the decision tree as a function of the meta-features and of the matrix,” are limitations that include a mathematical concept, (MPEP § 2106.04(a)(2) sub I), which is one of the groupings of abstract ideas. The claim also recites more details or specifics to the abstract idea of “[(e)] determining meta-features”, where “[(e.1)] the meta-features characterizing at least one of the following properties of the training data sets: image resolution, number of classes, number of video frames,” and of “[(g)] optimizing“ [(g.1)] so that the decision tree outputs which of the determined optimal parameterizations using BOHB is a suitable parameterization for the meta-features,” and “[(g.2)] wherein a subset of meta-features of the plurality of the meta-features is determined using a Greedy algorithm,” and accordingly, are merely more specific to the abstract idea. Thus, claim 1 recites an abstract idea.
Under Step 2A Prong Two, the claim as a whole is not integrated into a practical application, because the additional elements recited in the claim beyond the identified judicial exception include a “computer-implemented method,” and a “system,” which are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), and do not serve to integrate the abstract idea into a practical application. The claim also recites a “machine learning system,” which is recited at a high level of generality, and accordingly, is a generic computer component used to implement the abstract idea, (MPEP § 2106.05(f)), that does not integrate the abstract idea into a practical application. The claim also recites “[(a)] providing predefined hyperparameters,” and “[(f)] initializing a system including a decision tree,” which are insignificant extra-solution activities of mere data gathering, (MPEP § 2106.05(g)), that are pre-solution activities incidental to the primary process of creating a machine learning system for computer vision, and accordingly, do not integrate the abstract idea into a practical application. Also, the claim recites more details or specifics to the additional element of “[(a)] providing,” where the hyperparameters include “[(a.1)] one first parameter, which characterizes which optimization method is used,” and “[(a.2)] one second parameter, which characterizes of what type of the machine learning system is,” and accordingly, are merely more specific to the abstract idea. Therefore, claim 1 is directed to the abstract idea.
Finally, under Step 2B, the additional elements, taken alone or in combination, do not represent significantly more than the abstract idea itself, because the additional elements recited in the claim beyond the identified judicial exception include a “computer-implemented method,” and a “system,” which are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), and do not amount to significantly more than the abstract idea. The claim also recites a “machine learning system,” which is recited at a high level of generality, and accordingly, is a generic computer component used to implement the abstract idea, (MPEP § 2106.05(f)), that does not amount to significantly more than the abstract idea. The claim also recites “[(a)] providing predefined hyperparameters,” which is a well-understood, routine, and conventional activity of receiving and transmitting data over a network, (MPEP § 2106.05(d) sub II.i), that does not amount to significantly more than the abstract idea. The claim also recites “[(f)] initializing a system including a decision tree,” which is a well-understood, routine, and conventional activity of retrieving information in memory and storing information in memory, (MPEP § 2106.05(d) sub II.iv), that does not amount to significantly more than the abstract idea. Also, the claim recites more details or specifics to the additional element of “[(a)] providing,” where the hyperparameters include “[(a.1)] one first parameter, which characterizes which optimization method is used,” and “[(a.2)] one second parameter, which characterizes of what type of the machine learning system is,” and accordingly, are merely more specific to the abstract idea. Therefore, claim 1 is subject-matter ineligible.
Claim 2 depends from claim 1, and recites “[(g.1)] wherein parameters of the decision tree are optimized using Autofolio,” which is a generic computer component used in the expected and manner of algorithm selectin and configuration that is used to implement the abstract idea of the claim, (MPEP § 2106.05(f)), and does not serve to integrate the abstract idea into a practical application under Step 2A Prong Two, nor does it amount to significantly more than the abstract idea under Step 2B. Accordingly, claim 2 is subject-matter ineligible.
Claim 3 and 4 depends from claim 1, and recites more details or specifics to the abstract idea of “[(c)] assessing . . . using a normalized metric,” where (claim 3: “[(c.1)] an average value of the normalized metric across the plurality of training data sets is determined,” “[(c.2)] one of the determined parameterizations of the hyperparameters being selected using BOHB, whose normalized metric comes closest to the average value,” “[(c.3)] the normalized metric of the selected parameterization for all training data sets being added to the matrix”; claim 4: “[(c.1)] wherein an average value of the normalized metric across the plurality of the training data sets is determined,” “[(c.2)] one of the determined parameterizations of the hyperparameters being selected using BOHB, which for the normalized metric, exhibits on average for all training data sets a greatest improvement of the normalized metric compared to the average value of the normalized metric,” “[(c.3)] the normalized metric of the selected parameterization for all training data sets being added to the matrix”), and accordingly, are merely more specific to the abstract idea. Accordingly, claims 3 and 4 are subject-matter ineligible.
Claim 5 depends from claim 1. The claim recites more details or specifics to the abstract idea of “[(e)] determining meta-features,” wherein “[(e.2)] the decision tree determines a suitable parameterization as a function of the subset of the meta-features and of the matrix,” and accordingly, are merely more specific to the abstract idea. Accordingly, claim 5 is subject-matter ineligible.
Claim 6 depends from claim 1. The claim recites, in relation to a “[(h)] further training data set . . . provided,” the claim recites limitations of “[(h.1)] meta-features for the further training data set being determined,” and “[(h.2)] a suitable parameterization being subsequently determined using the decision tree as a function of the meta-features for the further training data set and of the matrix.” These limitations of “being determined” are activities that can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinion, and accordingly, are a mental process, (MPEP § 2106.05(f)(2) sub III), which is one of the groupings of abstract ideas. Accordingly, claim 6 recites an abstract idea.
The claim recites “[(h)] a further training data set is provided,” which is an insignificant extra-solution activity of mere data gathering, (MPEP § 2106.05(g)), that is a pre-solution activity incidental to the primary process of creating a machine learning system for computer vision, and accordingly, does not integrate the abstract idea into a practical application under Step 2A Prong Two. Under Step 2B, the limitations is a well-understood, routine, and conventional activity of receiving and transmitting data over a network, (MPEP § 2106.05(d) sub II.i), that does not amount to significantly more than the abstract idea. The claim also recites the limitation of “[(h.3)] the machine learning system being created based on the suitable parameterization and being trained on the further training data set,” which is the use of a generic computer component (machine learning system) to implement the abstract idea, (MPEP § 2106.05(f)), that does not serve to integrate the abstract idea into a practical application under Step 2A Prong Two, nor amounts to significantly more than the abstract idea under Step 2B. Accordingly, claim 6 is subject-matter ineligible.
Claim 7 depends from claim 1. The claim recites more details of the additional element of “[(a)] providing predefined hyperparameters,” wherein “[(a.3)] the machine learning system is created based on the first parameter,” and “[(a.4)] an optimization algorithm for the machine learning system is selected based on the second parameter and parameterized in accordance with the output suitable parameterization,” and accordingly, are merely more specific to the additional element. Also, these limitations of “created” and “selected” is the use of the generic computer component (machine learning system) to implement the abstract idea, (MPEP § 2106.05(f)), that does not integrate the abstract idea into a practical application under Step 2A Prong Two, and does not amount to significantly more than the abstract idea under Step 2B. Accordingly, claim 7 is subject- matter ineligible.
Claim 8 depends from claim 1. The claim recites more details of the additional element of “[(a)] providing predefined hyperparameters,” wherein “[(a.3)] wherein the hyperparameters further include parameters, which characterize: a batch size, and/or a number of data points to be used for training, and/or a learning rate, and/or a number of data points according to which an efficiency of the machine learning system is to be evaluated, and/or a relationship of parameters of the machine learning system that remain unchanged during training of the machine learning system and/or a weight decay,” and accordingly, are merely more specific to the additional element. Accordingly, claim 8 is subject-matter ineligible.
Claim 9 recites a non-transitory machine-readable memory medium, which is a product, and thus one of the statutory categories of patentable subject matter. (35 U.S.C. § 101).
However, under Step 2A Prong One, the claim recites the limitations of “[(b)] determining an optimal parameterization of the hyperparameters using BOHB (Bayesian optimization (BO) and Hyperband (HB)) for each single training data set of a plurality of different training data sets for computer vision,” “[(c)] assessing all of the optimal parameterizations on all training data sets of the plurality of different training data sets using a normalized metric,” “[(d)] creating a matrix, the matrix including the normalized metric for each of the optimal parameterizations and for each of the training data sets,” “[(e)] determining meta-features for each of the training data sets,” “[(g)] optimizing the decision tree as a function of the meta-features and of the matrix.” These limitations of “[(b),(e)] determining,” “[(c)] assessing,” “[(d)] creating a matrix,” and “[(g)] optimizing” can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinion, and accordingly, are a mental process, (MPEP § 2106.05(f)(2) sub III), which are one of the groupings of abstract ideas. Also, the limitations of “[(b)] determining an optimal parameterization of the hyperparameters using BOHB (Bayesian optimization (BO) and Hyperband (HB)) for each single training data set of a plurality of different training data sets for computer vision,” and “[(d)] creating a matrix, the matrix including the normalized metric for each of the optimal parameterizations and for each of the training data sets,” and “[(g)] optimizing the decision tree as a function of the meta-features and of the matrix,” are limitations that include a mathematical concept, (MPEP § 2106.04(a)(2) sub I), which is one of the groupings of abstract ideas. The claim also recites more details or specifics to the abstract idea of “[(e)] determining meta-features”, where “[(e.1)] the meta-features characterizing at least one of the following properties of the training data sets: image resolution, number of classes, number of video frames,” and of “[(g)] optimizing“ “[(g.1)] so that the decision tree outputs which of the determined optimal parameterizations using BOHB is a suitable parameterization for the meta-features,” and “[(g.2)] wherein a subset of meta-features of the plurality of the meta-features is determined using a Greedy algorithm,” and accordingly, are merely more specific to the abstract idea. Thus, claim 9 recites an abstract idea.
Under Step 2A Prong Two, the claim as a whole is not integrated into a practical application, because the additional elements recited in the claim beyond the identified judicial exception include a “non-transitory machine-readable memory medium on which is stored a computer program for creating a system, which is suitable for creating, in an automated manner, a machine learning system for computer vision, the computer program, when executed by a computer, causing the computer to perform,” ,” which are generic computer components including a computer program having a set of interactions that are used to implement the abstract idea, (MPEP § 2106.05(f)), and do not serve to integrate the abstract idea into a practical application. The claim also recites a “machine learning system,” which is recited at a high level of generality, and accordingly, is a generic computer component used to implement the abstract idea, (MPEP § 2106.05(f)), that does not integrate the abstract idea into a practical application. The claim also recites “[(a)] providing predefined hyperparameters,” and “[(f)] initializing a system including a decision tree,” which are insignificant extra-solution activities of mere data gathering, (MPEP § 2106.05(g)), that are pre-solution activities incidental to the primary process of creating a machine learning system for computer vision, and accordingly, do not integrate the abstract idea into a practical application. Also, the claim recites more details or specifics to the additional element of “[(a)] providing,” where the hyperparameters include “[(a.1)] one first parameter, which characterizes which optimization method is used,” and “[(a.2)] one second parameter, which characterizes of what type of the machine learning system is,” and accordingly, are merely more specific to the abstract idea. Therefore, claim 9 is directed to the abstract idea.
Finally, under Step 2B, the additional elements, taken alone or in combination, do not represent significantly more than the abstract idea itself, because the additional elements recited in the claim beyond the identified judicial exception include a “non-transitory machine-readable memory medium on which is stored a computer program for creating a system, which is suitable for creating, in an automated manner, a machine learning system for computer vision, the computer program, when executed by a computer, causing the computer to perform,” ,” which are generic computer components including a computer program having a set of interactions that are used to implement the abstract idea, (MPEP § 2106.05(f)), and do not amount to significantly more than the abstract idea. The claim also recites a “machine learning system,” which is recited at a high level of generality, and accordingly, is a generic computer component used to implement the abstract idea, (MPEP § 2106.05(f)), that does not amount to significantly more than the abstract idea. The claim also recites “[(a)] providing predefined hyperparameters,” which is a well-understood, routine, and conventional activity of receiving and transmitting data over a network, (MPEP § 2106.05(d) sub II.i), that does not amount to significantly more than the abstract idea. The claim also recites “[(f)] initializing a system including a decision tree,” which is a well-understood, routine, and conventional activity of retrieving information in memory and storing information in memory, (MPEP § 2106.05(d) sub II.iv), that does not amount to significantly more than the abstract idea. Also, the claim recites more details or specifics to the additional element of “[(a)] providing,” where the hyperparameters include “[(a.1)] one first parameter, which characterizes which optimization method is used,” and “[(a.2)] one second parameter, which characterizes of what type of the machine learning system is,” and accordingly, are merely more specific to the abstract idea. Therefore, claim 9 is subject-matter ineligible.
Claim 10 recites a “device,” which is a product, and thus one of the statutory categories of patentable subject matter. (35 U.S.C. § 101). The preamble of the claim also recites, “the method,” which is a process, and thus also one of the statutory categories of patentable subject matter. (35 U.S.C. § 101). As noted, the combination of an apparatus and a method of using the device is subject to a rejection under Section 112, as set out above in detail. For purposes of examination, the claim is considered as directed to a “device.”
However, under Step 2A Prong One, the claim recites the limitations of “[(b)] determining an optimal parameterization of the hyperparameters using BOHB (Bayesian optimization (BO) and Hyperband (HB)) for each single training data set of a plurality of different training data sets for computer vision,” “[(c)] assessing all of the optimal parameterizations on all training data sets of the plurality of different training data sets using a normalized metric,” “[(d)] creating a matrix, the matrix including the normalized metric for each of the optimal parameterizations and for each of the training data sets,” “[(e)] determining meta-features for each of the training data sets,” “[(g)] optimizing the decision tree as a function of the meta-features and of the matrix.” These limitations of “[(b),(e)] determining,” “[(c)] assessing,” “[(d)] creating a matrix,” and “[(g)] optimizing” can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinion, and accordingly, are a mental process, (MPEP § 2106.05(f)(2) sub III), which are one of the groupings of abstract ideas. Also, the limitations of “[(b)] determining an optimal parameterization of the hyperparameters using BOHB (Bayesian optimization (BO) and Hyperband (HB)) for each single training data set of a plurality of different training data sets for computer vision,” and “[(d)] creating a matrix, the matrix including the normalized metric for each of the optimal parameterizations and for each of the training data sets,” and “[(g)] optimizing the decision tree as a function of the meta-features and of the matrix,” are limitations that include a mathematical concept, (MPEP § 2106.04(a)(2) sub I), which is one of the groupings of abstract ideas. The claim also recites more details or specifics to the abstract idea of “[(e)] determining meta-features”, where “[(e.1)] the meta-features characterizing at least one of the following properties of the training data sets: image resolution, number of classes, number of video frames,” and of “[(g)] optimizing“ [(g.1)] so that the decision tree outputs which of the determined optimal parameterizations using BOHB is a suitable parameterization for the meta-features,” and “[(g.2)] wherein a subset of meta-features of the plurality of the meta-features is determined using a Greedy algorithm,” and accordingly, are merely more specific to the abstract idea. Thus, claim 10 recites an abstract idea.
Under Step 2A Prong Two, the claim as a whole is not integrated into a practical application, because the additional elements recited in the claim beyond the identified judicial exception include a “computer-implemented method,” and a “system,” which are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)),, and do not serve to integrate the abstract idea into a practical application. The claim also recites a “machine learning system,” which is recited at a high level of generality, and accordingly, is a generic computer component used to implement the abstract idea, (MPEP § 2106.05(f)), that does not integrate the abstract idea into a practical application. The claim also recites “[(a)] providing predefined hyperparameters,” and “[(f)] initializing a system including a decision tree,” which are insignificant extra-solution activities of mere data gathering, (MPEP § 2106.05(g)), that are pre-solution activities incidental to the primary process of creating a machine learning system for computer vision, and accordingly, do not integrate the abstract idea into a practical application. Also, the claim recites more details or specifics to the additional element of “[(a)] providing,” where the hyperparameters include “[(a.1)] one first parameter, which characterizes which optimization method is used,” and “[(a.2)] one second parameter, which characterizes of what type of the machine learning system is,” and accordingly, are merely more specific to the abstract idea. Therefore, claim 10 is directed to the abstract idea.
Finally, under Step 2B, the additional elements, taken alone or in combination, do not represent significantly more than the abstract idea itself, because the additional elements recited in the claim beyond the identified judicial exception include a “computer-implemented method,” and a “system,” which are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)),, and do not amount to significantly more than the abstract idea. The claim also recites a “machine learning system,” which is recited at a high level of generality, and accordingly, is a generic computer component used to implement the abstract idea, (MPEP § 2106.05(f)), that does not amount to significantly more than the abstract idea. The claim also recites “[(a)] providing predefined hyperparameters,” which is a well-understood, routine, and conventional activity of receiving and transmitting data over a network, (MPEP § 2106.05(d) sub II.i), that does not amount to significantly more than the abstract idea. The claim also recites “[(f)] initializing a system including a decision tree,” which is a well-understood, routine, and conventional activity of retrieving information in memory and storing information in memory, (MPEP § 2106.05(d) sub II.iv), that does not amount to significantly more than the abstract idea. Also, the claim recites more details or specifics to the additional element of “[(a)] providing,” where the hyperparameters include “[(a.1)] one first parameter, which characterizes which optimization method is used,” and “[(a.2)] one second parameter, which characterizes of what type of the machine learning system is,” and accordingly, are merely more specific to the abstract idea. Therefore, claim 10 is subject-matter ineligible.
Claim Rejections – 35 U.S.C. § 103
7. The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
8. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. § 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
9. This application currently names joint inventors. In considering patentability of the claims the Examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the Examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention.
10. Claims 1 and 6-10 are rejected under 35 U.S.C. § 103 as being unpatentable over US Published Application 20200380378 to Moharrer et al. [hereinafter Moharrer] in view of Falkner et al., "BOHP: Robust and Efficient Hyperparameter Optimization at Scale," and Supplementary Material, arXiv (2018) [hereinafter Falkner], US Published Application 20210357744 to Mittal et al. [hereinafter Mittal], and Katz et al., “ExploreKit: Automatic Feature Generation and Selection,” IEEE (2016) [hereinafter Katz].
Regarding claims 1, 9, and 10, Moharrer teaches [a] computer-implemented method for creating a system, which is suitable for creating, in an automated manner, a machine learning system for computer vision (Moharrer, claim 1, teaches a “method”; Moharrer ¶ 0037 teaches “all datasets within training corpus 170 have their own (e.g. different) values for a same set of metafeatures that describe a dataset as a whole, more or less without regard for any particular item in the dataset. For example, one metafeature may count how many colors are in a palette needed to render all photos in a dataset, which may have a smaller integer value if the dataset is monochromatic [(that is, the “training corpus” is for creating a system, which is suitable for creating, in an automated manner, a machine learning system for computer vision)]”) of claim 1, [a] non-transitory machine-readable memory mediums on which is stored a computer program for creating a system . . . , the computer program, when executed by a computer, causing the computer to perform (Moharrer, claim 10, teaches “[o]ne or more non-transitory computer-readable media storing instructions [(that is, a computer program)] that, when executed by one or more processors [(that is, a computer)]”) of claim 9, and [a] device configured to create a system, which is suitable for creating, in an automated manner, a machine learning system for computer vision (Moharrer ¶ 0089 teaches a “[c]omputer system 400 may implement the techniques described herein using customized hard-wired logic, one or more ASICs or FPGAs, firmware and/or program logic which in combination with the computer system causes or programs computer system 400 to be a special-purpose machine [(that is, a device configured to create a system)]”), the method comprising the following steps the method comprising the following steps:
[(a)] providing predefined hyperparameters (Moharrer, Fig. 1A, teaches a prediction of how effective a machine learning (ML) model be if configured with new values of hyperparameters [Examiner annotations in dashed-line text boxes]:
PNG
media_image1.png
654
961
media_image1.png
Greyscale
Moharrer ¶ 0016 teaches “several initial points are used to roughly survey a multidimensional hyperparameters space for configuring a machine learning (ML) model, before exploring regions with more promising sets of hyperparameters values [(that is, to “roughly survey a multidimensional hyperparameter space . . . before exploring” is providing predefined hyperparameters)]”; also, for example, Moharrer ¶ 0033 teaches that “if ML model 130 is a classifier, then ML model 130 may select one of multiple mutually exclusive labels (i.e. classifications) for the sample, such as hot and cold”), the hyperparameters including at least [(a.1)] one first parameter . . . , and [(a.2)] one second parameter, . . . (Moharrer ¶ 0030 teaches “Each configuration consists of a respective value for each of hyperparameters X-Y [(that is, the hyperparameters including at least one first parameter . . . and one second parameter )]”);
[(b)] determining an optimal parameterization of the hyperparameters . . . for each single training data set of a plurality of different training data sets for computer vision (Moharrer ¶ 0024 teaches “inferencing by the general metamodel for hyperparameters optimization of the ML model given a new dataset. Metafeatures about the new dataset are used with the trained metamodel to warm start a hyperparameters optimization algorithm”; Moharrer ¶ 0047 teaches an “example pseudocode may implement an example hyperparameter optimization.
PNG
media_image2.png
54
323
media_image2.png
Greyscale
PNG
media_image3.png
131
327
media_image3.png
Greyscale
Moharrer ¶ 0040 teaches that “[d]uring training, ML model 130 is (e.g. concurrently) repeatedly reconfigured and retrained, each time with a distinct pairing of a training dataset of 111-112 and a configuration of 121-122 [(that is, “each time” is determining an optimal parameterization of the hyperparameters . . . for each single training data set of a plurality of different training data sets for computer vision)]”);
[(c)] assessing all of the optimal parameterizations on all training data sets of the plurality of different training data sets using a . . . metric (Moharrer ¶ 0025 teaches “for each training dataset, a computer derives, from the training dataset, values for dataset metafeatures. . . . [O]btaining an empirical quality score [(that is, assessing . . . using a normalized metric)] that indicates how effective was said training the ML model when configured with the hyperparameters configuration”);
[(d)] creating a matrix, the matrix including the . . . metric for each of the optimal parameterizations and for each of the training data sets (Moharrer, Fig. 1A, teaches creating a matrix of performance tuples 140 [Examiner annotations in dashed-line text boxes]:
PNG
media_image1.png
654
961
media_image1.png
Greyscale
Moharrer ¶ 0041 teaches “[w]ith different scores for different datasets [(that is, “training datasets 111 and 112” is for each of the training datasets)] and for different hyperparameters values [(that is, for each of the optimal parameterizations)], performance tuples 140 may be generated based on training corpus 170 [having training datasets 111 and 112] and configurations 180 [having parameter X and parameter Y (that is, one first parameter and one second parameter)]. When ignoring shown demonstrative header rows, for example, first data row of [training] corpus 170 and first data row of configurations 180 are combined with the quality score that they achieved together [(that is, the . . . metric)], which forms first data row of performance tuples 140”);
[(e)] determining meta-features for each of the training data sets (Moharrer, Fig. 1B, teaches
PNG
media_image4.png
655
1014
media_image4.png
Greyscale
Moharrer ¶ 0023 teaches “[d]ataset metafeatures are obtained for each dataset [111 and 112] and combined with hyperstars, a current hyperparameters configuration, and a corresponding performance score”), . . . ;
[(f)] initializing the system including a decision tree (Moharrer ¶ 0113 teaches “[c]lasses of problems that machine learning (ML) excels at include clustering, classification, regression, anomaly detection, prediction, and dimensionality reduction (i.e. simplification). Examples of machine learning algorithms include decision trees, . . . . Implementations of machine learning may rely on matrices”; with regard to machine learning models”, with regard to initialization, Moharrer ¶ 0137 teaches “[r]andom forest hyper-parameters may include: number-of-trees-in-the-forest, maximum-number-of-features-considered-for-splitting-a-node, number-of-levels-in-each-decision-tree, minimum-number-of-data-points-on-a-leaf-node, method-for-sampling-data-points, etc.”; with respect to a cold-start, in a general overview, Moharrer ¶ 0016 teaches that “[t]o prepare a metamodel (i.e. meta-learning hyperparameters optimizer) for a cold start (i.e. untrained meta-model or unfamiliar dataset), several initial points [(that is, “initial points” is initializing a system including a decision tree)] are used to roughly survey a multidimensional hyperparameters space for configuring a machine learning (ML) model [such as a decision tree], before exploring regions with more promising sets of hyperparameters values.”); and
[(g)] optimizing the decision tree as a function of the meta-features and of the matrix (Moharrer, Algorithm 3, teaches “Hyperparameter Optimization with MetaLearning,” with regard to ML model A [(that is, decision tree)]:
PNG
media_image5.png
287
631
media_image5.png
Greyscale
PNG
media_image6.png
274
404
media_image6.png
Greyscale
Moharrer, Algorithm 3 & Fig. 1B above, teaches “Snew ← Extract the metafeatures of [new dataset] Dnew,” [(that is, the meta-features )] and “Lnew ← Extract the landmark features [(that is, “metafeatures” and “landmark features” is the matrix)] corresponding to [dataset] Dnew and [ML model] A [(that is, decision tree)])”
[(g.1)] so that the decision tree outputs which of the determined optimal parameterizations (Moharrer, Algorithm 3, teaches “Output: Best Hyperparameter configuration H* [(that is, [(g.1)] so that the decision tree outputs which of the determined optimal parameterization)]”) . . . .
Though Moharrer teaches hyperparameter having at least a first and a second parameter, as well as a machine learning model may have one of many defined model types and types of hyperparameter optimization, Moharrer, however, does not explicitly teach –
* * *
[(a) providing predefined hyperparameters, the hyperparameters including at least (a.1) one first parameter], which characterizes which optimization method is used, and [(a.2) one second parameter], which characterizes of what type of the machine learning system is;
[(b) determining an optimal parameterization of the hyperparameters] using BOHB (Bayesian Optimization (BO) and Hyperband (HB)) [for each single training data set of a plurality of different training data sets for computer vision];
* * *
[(g) and optimizing the decision tree (g.1) so that the decision tree outputs which of the determined optimal parameterizations] using BOHB is a suitable parameterization for the meta-features.
But Falkner teaches –
* * *
[(a) providing predefined hyperparameters, the hyperparameters including at least (a.1) one first parameter], which characterizes which optimization method is used, and [(a.2) one second parameter], which characterizes of what type of the machine learning system is (Falkner, left column of p. 3, “3. Bayesian Optimization and Hyperband,” first paragraph, teaches “right column of p. 1, “1. Introduction, 4. Scalability,” first paragraph, teaches “[m]odern deep neural networks require the setting of a multitude of hyperparameters, including architectural choices (e.g., the number and width of layers) [(that is, the hyperparameters including at least: . . . one second parameter, which characterizes of what type of the machine learning system is)], optimization hyperparameters (e.g., learning rate schedules, momentum, and batch size) [(that is, the hyperparameters including at least: one first parameter, which characterizes which optimization method is used)], and regularization hyperparameters (e.g., weight decay and dropout rates)”);
[(b) determining an optimal parameterization of the hyperparameters] using BOHB (Bayesian Optimization (BO) and Hyperband (HB)) [for each single training data set of a plurality of different training data sets for computer vision] (Falkner, right column of p. 3, “4. Model-Based Hyperband,” first paragraph, teaches “introduce our new practical HPO method, which we dub BOHB since it combines Bayesian optimization (BO) and Hyperband (HB). We designed BOHB to satisfy all the desiderata described in the introduction. HB already satisfies most of these desiderata (in particular, strong anytime performance, scalability, robustness and flexibility), and we combine it with BO to also satisfy the desideratum of strong final performance in BOHB”; Falkner, right column of p. 8, “5.5 Convolutional Neural Networks on CIFAR-10,” first paragraph, teaches “[t]o perform hyperparameter optimization, we split off 5000 training images as a validation set [(that is, “CIFAR-10” is an image dataset for training image classification models, which is a training data set for computer vision)]”);
* * *
[(g) and optimizing the decision tree (g.1) so that the decision tree outputs which of the determined optimal parameterizations] using BOHB is a suitable parameterization for the meta-features (see above, Falkner, right column of p. 3, “4. Model-Based Hyperband,” first paragraph, which teaches “introduce our new practical [hyperparameter optimization (HPO)] method, which we dub BOHB since it combines Bayesian optimization (BO) and Hyperband (HB) [(that is, [optimizing] using BOHB is a suitable parametrization for the meta-features)]”).
Moharrer and Falkner are from the same or similar field of endeavor. Moharrer teaches techniques that train regressor(s) to predict how effective would a machine learning model (MLM) be if trained with new hyperparameters and/or dataset. Falkner teaches hyperparameter optimization method that is conceptually simple and easy to implement. Thus, it would have been obvious to a person having ordinary skill in the art as of the effective filing date of the Applicant’s invention to modify Moharrer pertaining to machine learning model hyperparameter optimization with the Bayesian optimization and Hyperband (BOHB) hyperparameter optimization of Falkner.
The motivation to do so is for the use of a “hyperparameter optimization method, which consistently outperforms both Bayesian optimization and Hyperband on a wide range of problem types, . . . [The] method is robust and versatile, while at the same time being conceptually simple and easy to implement.” (Falkner, Abstract).
Though Moharrer and Falkner teach the feature of assessing performance scores as a “metric,” inherently, such assessment is meaningful when the scores are on a “normalized” basis as needed, in which the “quality scores” of Moharrer are within a normalized score of 0 < x < 1, the combination of. Moharrer and Falkner, however, do not explicitly teach “[(c) assessing . . . using a] normalized metric.” Also, the combination of Moharrer and Falkner do not explicitly teach –
* * *
[(e) determining meta-features for each of the training data sets, (e.1)] the meta-features characterizing at least one of the following properties of the training data sets: image resolution, number of classes, number of video frames;
* * *
But Mittal teaches “[(c) assessing . . . using a] normalized metric.” (Mittal ¶ 0031 teaches, that, “for each of the plurality of tasks and each of the plurality of hyperparameter configurations, the system determines a performance score indicating a performance achieved for the corresponding hyperparameter configuration. . . . In some embodiments, the determination of a performance score further includes normalizing the raw scores based on a mean and a standard deviation over the raw scores. The normalized scores are then used as the performance scores in the joint space of tasks and hyperparameter configurations [(that is, assessing . . . a normalized metric)]).
Mittal further teaches –
* * *
[(e) determining meta-features for each of the training data sets, (e.1)] the meta-features characterizing at least one of the following properties of the training data sets: image resolution, number of classes, number of video frames (Mittal ¶ 0055 teaches “Jointly training the performance prediction network and the representations in an end-to-end fashion constitutes a departure from previous meta learning approaches that represent a task using hand-crafted metadata ( e.g., total number of training samples, number of classes, number of samples per class, etc.) [(that is, the meta -features characterizing at least one of the following properties of the training data sets: . . . number of classes)]”);
* * *
Moharrer, Falkner, and Mittal are from the same or similar field of endeavor. Moharrer teaches techniques that train regressor(s) to predict how effective would a machine learning model (MLM) be if trained with new hyperparameters and/or dataset. Falkner teaches hyperparameter optimization method that is conceptually simple and easy to implement. Mittal teaches a task-aware recommendation of hyperparameter configurations for a neural network architecture implementing hyperparameter optimization. Thus, it would have been obvious to a person having ordinary skill in the art as of the effective filing date of the Applicant’s invention to modify the combination of Moharrer and Falkner pertaining to machine learning model hyperparameter optimization using a Bayesian optimization and Hyperband (BOHB) hyperparameter optimization with the metric normalization of Mittal.
The motivation to do so is because “Automated Machine Learning (AutoML) is a machine learning paradigm which automates the process of searching neural network architectures, via Neural Architecture Search (NAS) and associated hyperparameters, via Hyperparameter Optimization systems (HPO), with little or no manual intervention.” (Mittal ¶ 0003).
Though Moharrer, Falkner, and Mittal teach that subsequent model configurations may be greedily based on previous configurations, the combination of Moharrer, Falkner, and Mittal does not explicitly teach–
* * *
[(g)] optimizing the decision tree] . . . [(g.2)] wherein a subset of meta-features of the plurality of the meta-features is determined using a Greedy algorithm.
But Katz teaches -
* * *
[(g)] optimizing the decision tree] . . . [(g.2)] wherein a subset of meta-features of the plurality of the meta-features is determined using a Greedy algorithm (Katz, left column of p. 980, “IV. The Proposed Method,” first & second paragraphs, teaches “a novel ML based approach for candidate feature ranking. We define meta-features to represent both the dataset and the candidate feature and train a feature ranking classifier. . . . In the candidate features evaluation & selection phase we use greedy search [(that is, using a Greedy algorithm)] to evaluate the ranked candidate features. We evaluate the performance of the joint set
F
i
∪
f
i
,
j
c
a
n
d
for each
f
i
,
j
c
a
n
d
∈
R
a
n
k
e
d
F
i
c
a
n
d
and computer the reduction in clasifrication error compared with Fi. When the performance improvement exceeds a predefined threshold ϵw, the evaluation process terminates and we select the current candidate feature, denoted as
f
i
s
e
l
e
c
t
[(that is, a subset of meta-features of the plurality of the meat-features is determined using a Greedy algorithm)]”)
Moharrer, Falkner, Mittal and Katz are from the same or similar field of endeavor. Moharrer teaches techniques that train regressor(s) to predict how effective would a machine learning model (MLM) be if trained with new hyperparameters and/or dataset. Falkner teaches hyperparameter optimization method that is conceptually simple and easy to implement. Mittal teaches a task-aware recommendation of hyperparameter configurations for a neural network architecture implementing hyperparameter optimization. Katz teaches a framework for automated feature generation for a training dataset that includes meta-features.
Thus, it would have been obvious to a person having ordinary skill in the art as of the effective filing date of the Applicant’s invention to modify the combination of Moharrer, Falkner and Mittal pertaining to machine learning model hyperparameter optimization using a Bayesian optimization and Hyperband (BOHB) hyperparameter optimization with the automated feature generation of Katz.
The motivation to do so is because such an “approach enables efficient identification of the new features and produces superior results compared to existing feature selection solutions.” (Katz, Abstract).
Regarding claim 6, the combination of Moharrer, Falkner, Mittal and Katz teach all of the limitations of claim 1, as described above in detail.
Moharrer teaches -
[(h)] wherein a further training data set is provided (Moharrer, Fig. 1B, teaches a new dataset 113 [Examiner annotations in dashed-line text boxes];
PNG
media_image7.png
742
1013
media_image7.png
Greyscale
Moharrer ¶ 0056 teaches “regressor 150 may make predictions for a new dataset, such as 113 [(that is, a further training data set is provided)]”),
[(h.1)] meta-features for the further training data set being determined (Moharrer ¶ 0056 teaches “[i]nput tuples for inferencing may include metafeatures of new dataset 113, hyperparameters of ML model 130, and landmarks scores as shown [(that is, “inferencing metafeatures” is meta-features for the further training data set being determined)]”),
[(h.2)] a suitable parameterization being subsequently determined . . . as a function of the meta-features for the further training data set and of the matrix (Moharrer ¶ 0056 teaches “regressor 150 may make predictions for . . . new hyperparameter configurations. Input tuples for inferencing may include metafeatures of new dataset 113, hyperparameters of ML model 130, and landmarks scores as shown”),
[(h.3)] the machine learning system being created based on the suitable parameterization and being trained on the further training data set (Moharrer ¶ 0032 teaches “[t]he lifecycle of ML model 130 has two phases. The first phase is preparatory and entails training as shown, such as in a laboratory. The second phase entails inferencing (not shown) in a production environment, such as with live and/or streaming data [(that is, the “production environment” is ”the machine learning system being created)]; Moharrer ¶ 0056 teaches “while in a production environment, processing of new production dataset 113 may begin by actually training ML model 130 with new dataset 113 and landmark configurations 1-L to generate the landmarks scores shown in the production input table [(that is, based on the suitable parameterization and being trained on the further training data set)]”).
Falkner teaches that “[Bayesian optimization (BO)] uses an acquisition function a : X → R based on the current model p(f|D) that trades off exploration and exploitation [of the hyperparameter configuration space X]. Based on the model and the acquisition function, it iterates the following three steps: (1) select the point that maximizes the acquisition function xnew = arg maxxϵX a(x), (2) evaluate the objective function ynew = f(xnew)+ϵ, and (3) augment the data D ← D ∪ (xnew, ynew) and refit the model.” (Falkner, left column at p. 3, “3.1 Bayesian Optimization,” first paragraph). Further, Falkner teaches a “Tree Parzen Estimator (TPE) . . . is a Bayesian optimization method [(that is, suitable parameterization] that uses a kernel density estimator to model the densities . . . over the input configuration space [of hyperparameters] . . . .” (Falkner, left column at p. 3, “3.1 Bayesian Optimization, Tree Parzen Estimator (TPE),” first paragraph).
[Examiner notes, in view of Falkner, the “Tree Parzen Estimator (TPE)” is a Bayesian optimization algorithm used for hyperparameter optimization that models the relationship between hyperparameter settings and model performance using probability distributions (“densities” in Falkner), specifically focusing on "good" and "bad" configurations to find the optimal hyperparameter set].
Regarding claim 7, the combination of Moharrer, Falkner, Mittal and Katz teach all of the limitations of claim 1, as described above in detail.
Falkner teaches -
wherein the machine learning system is created based on the first parameter, and an optimization algorithm for the machine learning system is selected based on the second parameter (Falkner, left column of p. 3, “3. Bayesian Optimization and Hyperband,” first paragraph, teaches “right column of p. 1, “1. Introduction, 4. Scalability,” first paragraph, teaches “[m]odern deep neural networks require the setting of a multitude of hyperparameters, including architectural choices (e.g., the number and width of layers) [(that is, an optimization algorithm for the machine learning system is selected based on the second parameter)], optimization hyperparameters (e.g., learning rate schedules, momentum, and batch size) [(that is, the machine learning system is created based on the first parameter)], and regularization hyperparameters (e.g., weight decay and dropout rates)”) and
parameterized in accordance with an output suitable parameterization (Falkner, left column of p. 3, “3. Bayesian Optimization and Hyperband,” first paragraph, teaches “[t]he hyperparameter optimization (HPO) problem is then defined as finding x* ϵ arg minx ϵ X f(x) [(that is, the “HPO problem” is parameterized in accordance with the output suitable parameterization)]”).
Regarding claim 8, the combination of Moharrer, Falkner, Mittal and Katz teach all of the limitations of claim 1, as described above in detail.
Moharrer teaches -
[(a.3)] wherein the hyperparameters further include parameters, which characterize: a batch size, and/or a number of data points to be used for training (with regard to “hyperstars,” Moharrer ¶ 0049 & Fig. 1B teaches that “[w]hile a training performance tuple [140] having a quality score, dataset meta-features, and a hyperparameter configuration provides good information, the tuple insufficiently describe a natural spectrum of possible operational performance, such as quality scores possible with other hyperparameter configurations and/or other datasets for a same ML model. Herein, hyperstars are points within a configuration hyperspace that serve as landmarks, such as 185 [(that is, “hyperstars” and “landmarks” are the hyperparameters further include parameters, which characterize; . . . a number of data to be used for training)], and are introduced to make quality score prediction flexible and more accurate for a regressor such as 150), and/or
a learning rate, and/or
a number of data points according to which an efficiency of the machine learning system is to be evaluated, and/or
a relationship of parameters of the machine learning system that remain unchanged during training of the machine learning system and/or
a weight decay.
11. Claim 2 is rejected under 35 U.S.C. § 103 as being unpatentable over US Published Application 20200380378 to Moharrer et al. [hereinafter Moharrer] in view of Falkner et al., "BOHP: Robust and Efficient Hyperparameter Optimization at Scale," and Supplementary Material, arXiv (2018) [hereinafter Falkner], US Published Application 20210357744 to Mittal et al. [hereinafter Mittal], Katz et al., “ExploreKit: Automatic Feature Generation and Selection,” IEEE (2016) [hereinafter Katz], and Lindauer et al., “AutoFolio: An Automatically Configured Algorithm Selector,” JAIR (2015) [hereinafter Lindauer].
Regarding claim 2, the combination of Moharrer, Falkner, Mittal and Katz teach all of the limitations of claim 1, as described above in detail.
Though Moharrer, Falkner, Mittal and Katz teach the features of generating meta-features and performance metrics in a matrix for hyperparameter optimization for a decision tree, the combination of Moharrer, Falkner, Mittal and Katz, however, does not explicitly teach -
wherein parameters of the decision tree are optimized using AutoFolio.
But Lindauer teaches –
wherein parameters of the decision tree are optimized using AutoFolio (Lindauer at p. § 10, “3. Configuration of Algorithm Selectors,” first paragraph, teaches, “AutoFolio approach of using algorithm configurators to automatically customize flexible algorithm selection (AS) frameworks to specific AS scenarios [(that is, in view of Moharrer ¶ 0113 above, a generic “specific algorithm selector scenarios” is the decision tree)]”; see ; Lauder at p. 9, “3.1 Formal Problem Statement,” last partial paragraph, teaches ‘to still be able to optimize parameters [(that is, parameters of the decision tree are optimized)] without access to that test set, the standard solution in machine learning is to partition the training set further, into k cross-validation [(CV)] folds”; first paragraph, teaches “Let
D
t
r
a
i
n
(
1
)
; : :
D
t
r
a
i
n
(
k
)
be a random partition of the training set Dtrain. The cross-validation performance CV (Sc) by of [algorithm selector] Sc on the training set is then:
PNG
media_image8.png
88
573
media_image8.png
Greyscale
In the end, we optimize the performance CV (Sc) by determining a configuration
c
^
∈
C
of the selector S with good cross-validation performance
PNG
media_image9.png
50
432
media_image9.png
Greyscale
and evaluate
c
^
by training a selector
S
c
^
^
with it on the entire training data and evaluating
P
(
S
c
^
)
on Dtest, as defined in Equation 1.
PNG
media_image10.png
63
476
media_image10.png
Greyscale
Lindauer at p. 11, “3.1 Formal Problem Statement,” second paragraph, teaches “we use each of the k folds
D
t
r
a
i
n
{
j
)
as one instance within the configuration process. . . . We note that many configurators . . . can discard configurations when they perform poorly on a subset of meta-instances and therefore do not have to evaluate all k cross-validation folds for every configuration. This saves time and lets us evaluate more congurations within the same configuration budget. Based on these considerations, Algorithm 1 [AutoFolio: Automated configuration of an algorithm selector] outlines the process to configure an algorithm selector with AutoFolio [(that is, wherein parameters of the decision tree are optimized using AutoFolio)]).
Moharrer, Falkner, Mittal, Katz, and Lindauer are from the same or similar field of endeavor. Moharrer teaches techniques that train regressor(s) to predict how effective would a machine learning model (MLM) be if trained with new hyperparameters and/or dataset. Falkner teaches hyperparameter optimization method that is conceptually simple and easy to implement. Mittal teaches a task-aware recommendation of hyperparameter configurations for a neural network architecture implementing hyperparameter optimization. Katz teaches a framework for automated feature generation for a training dataset that includes meta-features. Lindauer teaches implements a large variety of different [algorithm selection (AS)] approaches and their respective parameters in a single, highly-parameterized algorithm framework.
Thus, it would have been obvious to a person having ordinary skill in the art as of the effective filing date of the Applicant’s invention to modify the combination of Moharrer, Falkner, Mittal, and Katz pertaining to machine learning model hyperparameter optimization of a decision tree with the AutoFolio of Lindauer.
The motivation to do so is because “AutoFolio[] allows researchers and practitioners across a broad range of applications to exploit the combined power of many different AS methods.” (Lindauer, Abstract).
12. Claim 3 is rejected under 35 U.S.C. § 103 as being unpatentable over US Published Application 20200380378 to Moharrer et al. [hereinafter Moharrer] in view of Falkner et al., "BOHP: Robust and Efficient Hyperparameter Optimization at Scale," and Supplementary Material, arXiv (2018) [hereinafter Falkner], US Published Application 20210357744 to Mittal et al. [hereinafter Mittal], Katz et al., “ExploreKit: Automatic Feature Generation and Selection,” IEEE (2016) [hereinafter Katz], and US Published Application 20220180209 to Xu et al. [hereinafter Xu].
Regarding claim 3, the combination of Moharrer, Falkner, Mittal, and Katz teach all of the limitations of claim 1, as described above in detail.
Mittal teaches a “normalized metric”, as set out above.
Mittal also teaches -
[(c.1)] wherein an average value of the normalized metric across the plurality of training data sets is determined (Mittal ¶ 0038 teaches “for each of the one or more batches of sampled datasets [(that is, across the plurality of training data sets)], a performance score may be determined for each hyperparameter configurations, and the one or more performance scores may be averaged to predict the overall performance score of each corresponding hyperparameter configuration [(that is, an average value of the normalized metric across the plurality of training data sets is determined)],” . . . .
Moharrer teaches -
. . . [(c.3)] the normalized metric of the selected parameterization for all training data sets being added to the matrix (Moharrer ¶ 0055 teaches “[p]erformance tuples 140 may be generated as discussed above, with some additional data population [(that is, being added to the matrix)]. Each row (i.e. tuple) of performance tuples 140 stores the respective quality score [(that is, normalized metric)] of each landmark, shown as landmark scores I-L in performance tuples 140”).
Falkner teaches using a Bayesian optimization and Hyperband (BOHB) hyperparameter optimization, as discussed above in detail.
Though Moharrer, Falkner, Mittal, and Katz teach the feature of a target performance score in an AutoML process, the combination of Moharrer, Falkner, Mittal, and Katz, however, does not explicitly teach -
. . . [(c.2)] one of the determined parameterizations of the hyperparameters being selected . . . , whose . . . metric comes closest to the average value, . . .
But Xu teaches -
. . . [(c.2)] one of the determined parameterizations of the hyperparameters being selected . . . , whose . . . metric comes closest to the average value (Xu ¶ 0054 teaches “[d]uring each training, the difference between the value predicted by the current AI model and an actual target value is determined by using a loss function, to update parameters [(that is, one of the determined parameterizations of the hyperparameters being selected)] of the AI model. When the AI model can predict the actually desired target value [(that is, one of the determined parameterizations . . . whose normalized metric)] or a value that is quite close to the actually desired target value [(that is, “quite close” being closest to the average value)], it is considered that the training of the AI model is completed [(that is, one of the determined parameterizations of the hyperparameters being selected . . . , whose . . . metric comes closes to the average value)]”, . . .
[Examiner notes that the “average value” is a target value of the hyperparameter optimization, and under a broadest reasonable interpretation covers the teachings of Xu pertaining to an “actually desired target value,” which is not inconsistent with the Applicant’s specification (MPEP § 2111)]),
Moharrer, Falkner, Mittal, Katz, and Xu are from the same or similar field of endeavor. Moharrer teaches techniques that train regressor(s) to predict how effective would a machine learning model (MLM) be if trained with new hyperparameters and/or dataset. Falkner teaches hyperparameter optimization method that is conceptually simple and easy to implement. Mittal teaches a task-aware recommendation of hyperparameter configurations for a neural network architecture implementing hyperparameter optimization. Katz teaches a framework for automated feature generation for a training dataset that includes meta-features. Xu teaches an AutoML system that determines, based on a task target, an initial AI model used to implement the task target for the user, the AutoML system provides an optimization manner of the trained AI model for the user based on the analysis result.
Thus, it would have been obvious to a person having ordinary skill in the art as of the effective filing date of the Applicant’s invention to modify the combination of Moharrer, Falkner, Mittal, and Katz pertaining to machine learning model hyperparameter optimization of a decision tree with the task target of Xu.
The motivation to do so is because, “[b]efore training is performed by using the data in the first data set or the second data set, the data in the data set is preprocessed, so that the data is more suitable for AI model training, thereby improving efficiency of AI model training and prediction accuracy of the AI model obtained through training by using the data.” (Xu ¶ 0021).
13. Claim 4 is rejected under 35 U.S.C. § 103 as being unpatentable over US Published Application 20200380378 to Moharrer et al. [hereinafter Moharrer] in view of Falkner et al., "BOHP: Robust and Efficient Hyperparameter Optimization at Scale," and Supplementary Material, arXiv (2018) [hereinafter Falkner], US Published Application 20210357744 to Mittal et al. [hereinafter Mittal], Katz et al., “ExploreKit: Automatic Feature Generation and Selection,” IEEE (2016) [hereinafter Katz], and US Published Application 20210304055 to Qi et al. [hereinafter Qi].
Regarding claim 4, the combination of Moharrer, Falkner, Mittal, and Katz teach all of the limitations of claim 1, as described above in detail.
Mittal teaches a “normalized metric”, as set out above.
Mittal also teaches -
[(c.1)] wherein an average value of the normalized metric across the plurality of the training data sets is determined (Mittal ¶ 0038 teaches “for each of the one or more batches of sampled datasets [(that is, across the plurality of training data sets)], a performance score may be determined for each hyperparameter configurations, and the one or more performance scores may be averaged to predict the overall performance score of each corresponding hyperparameter configuration [(that is, an average value of the normalized metric across the plurality of training data sets is determined)]”), . . . .
Moharrer teaches -
. . . [(c.3)] the normalized metric of the selected parameterization for all training data sets being added to the matrix (Moharrer ¶ 0055 teaches “[p]erformance tuples 140 may be generated as discussed above, with some additional data population [(that is, being added to the matrix)]. Each row (i.e. tuple) of performance tuples 140 stores the respective quality score [(that is, normalized metric)] of each landmark, shown as landmark scores I-L in performance tuples 140”).
Falkner teaches using a Bayesian optimization and Hyperband (BOHB) hyperparameter optimization, as discussed above in detail.
Though Moharrer, Falkner, Mittal and Katz teach the feature of a target performance score in an AutoML process, the combination of Moharrer, Falkner, Mittal, and Katz, however, does not explicitly teach -
. . . [(c.2)] one of the determined parameterizations of the hyperparameters being selected . . . , which for the normalized metric, exhibits on average for all training data sets a greatest improvement of the normalized metric compared to the average value of the normalized metric, . . . .
But Qi teaches -
. . . [(c.2)] one of the determined parameterizations of the hyperparameters being selected . . . , which for the normalized metric, exhibits on average for all training data sets a greatest improvement of the normalized metric compared to the average value of the normalized metric (Qi ¶ 0043 teaches “[t]he values generated by the function of the comparison of the performance metrics may be compared to one or more threshold values [(that is, “one or more threshold values” is the average value of the normalized metric)] to determine if a significant enough improvement in performance has been achieved [(that is, exhibits on average for all training data sets a greatest improvement of the normalized metric compared to the average value of the normalized metric)]"), . . . .
[Examiner notes that the “average value” is a target value of the hyperparameter optimization, and under a broadest reasonable interpretation covers the teachings of Xu pertaining to an “actually desired target value,” which is not inconsistent with the Applicant’s specification (MPEP § 2111)]),
Moharrer, Falkner, Mittal, Katz, and Qi are from the same or similar field of endeavor. Moharrer teaches techniques that train regressor(s) to predict how effective would a machine learning model (MLM) be if trained with new hyperparameters and/or dataset. Falkner teaches hyperparameter optimization method that is conceptually simple and easy to implement. Mittal teaches a task-aware recommendation of hyperparameter configurations for a neural network architecture implementing hyperparameter optimization. Katz teaches a framework for automated feature generation for a training dataset that includes meta-features. Qi teaches optimizing an automated machine learning (AutoML) operation to configure parameters of a machine learning model.
Thus, it would have been obvious to a person having ordinary skill in the art as of the effective filing date of the Applicant’s invention to modify the combination of Moharrer, Falkner, Mittal and Katz pertaining to machine learning model hyperparameter optimization of a decision tree with determining if a significant enough improvement in performance has been achieved in Qi.
The motivation to do so is to “perform automated improvement of the hyperparameter sampling in the AutoML process, based on historical performance, on a continuous or periodic basis.” (Qi ¶ 0021).
14. Claim 5 is rejected under 35 U.S.C. § 103 as being unpatentable over US Published Application 20200380378 to Moharrer et al. [hereinafter Moharrer] in view of Falkner et al., "BOHP: Robust and Efficient Hyperparameter Optimization at Scale," and Supplementary Material, arXiv (2018) [hereinafter Falkner], US Published Application 20210357744 to Mittal et al. [hereinafter Mittal], Katz et al., “ExploreKit: Automatic Feature Generation and Selection,” IEEE (2016) [hereinafter Katz], and Feurer et al., “Towards further Automation in AutoML,” ICML (2018) [hereinafter Feurer].
Regarding claim 5, the combination of Moharrer, Falkner, Mittal, and Katz teach all of the limitations of claim 1, as described above in detail.
Though Moharrer, Falkner, Mittal, and Katz teach the features of hyperparameter optimization in an AutoML environment, the combination of Moharrer, Falkner, Mittal, and Katz, however, does not explicitly teach –
[(e.2)] wherein the decision tree determines a suitable parameterization as a function of the subset of the meta-features and of the matrix.
But Feurer teaches -
[(e.2)] wherein the decision tree determines a suitable parameterization as a function of the subset of the meta-features and of the matrix (Feurer at p. 4, “4.2 Constructing a Portfolio of Specialized AutoML Systems,” third paragraph, teaches “[g]iven a set of AutoML policies, we can use dataset meta-features to select, on a per dataset basis [(that is, “selected meta-features” being the subset, which is as a function of the subset of the meta-features)], which policy to use. For this, we loosely follow the selector design of Hydra (Xu et al., 2010, 2011): for each pair of AutoML policies, we fit a random forest [(that is, decision tree)] to predict whether playing policy πA outperforms playing policy πB given dataset meta-features [(that is, with “”policy πA “ and “policy πB,” the decision tree determines a suitable parameterization)]”; Feurer at p. 5, “5.1 Setup,” second paragraph, teaches “[w]e then ran the cross-product of all 871 ML pipelines and 437 datasets [(that is, a plurality of data sets)] with both holdout and 5-fold cross-validation to obtain four performance matrices. Having access to these, we were able to run all experiments as table lookups [(that is, as a function . . . of the matrix)]”).
Moharrer, Falkner, Mittal, Katz, and Feurer are from the same or similar field of endeavor. Moharrer teaches techniques that train regressor(s) to predict how effective would a machine learning model (MLM) be if trained with new hyperparameters and/or dataset. Falkner teaches hyperparameter optimization method that is conceptually simple and easy to implement. Mittal teaches a task-aware recommendation of hyperparameter configurations for a neural network architecture implementing hyperparameter optimization. Katz teaches a framework for automated feature generation for a training dataset that includes meta-features. Feurer teaches, with new hyper-hyperparameters that include the choice of the evaluation strategy used in the loss function, time budges to use, and the optimization strategy, the possibility of automating these choices, that this improves over picking affixed strategy and that for different time horizons different strategies are necessary.
Thus, it would have been obvious to a person having ordinary skill in the art as of the effective filing date of the Applicant’s invention to modify the combination of Moharrer, Falkner, Mittal and Katz pertaining to machine learning model hyperparameter optimization of a decision tree with the random forest to determine a suitable parameterization of Feurer.
The motivation to do so is because “it is possible to improve over picking a fixed combination of evaluation strategy and portfolio of ML pipelines (which we will refer to as policy). Moreover, we show that it is possible to build a portfolio of policies with the greedy algorithm which performs up to 10% better than choosing the single best policy for a time horizon from the test set.” (Feurer at p. 1, “1. Introduction,” second paragraph).
Response to Arguments
15. Examiner has fully considered the Applicant’s arguments, and responds below accordingly.
Section 101
16. “Applicant submits that at the very least the following claim limitations do not [recite a judicial exception]:
• "[(b)] determining an optimal parameterization of the hyperparameters using BOHB (Bayesian optimization (BO) and Hyperband (HB)) for each single training data set of a plurality of different training data sets for computer vision"
• "[(g)] optimizing the decision tree as a function of the meta-features and of the matrix, [(g.1)] so that the decision tree outputs which of the determined optimal parameterizations using BOHB is a suitable parameterization for the meta-features"
(Response at p. 7).
Applicant submits that the claim limitation of “[(b)] determining an optimal parameterization . . .” does not recite a mental process. First, the ‘determining’ limitation does not fit the definition of a mental process. . . . The Patent Office had an obligation to make an argument establishing that the use of BOHB is one of these exercises of human reflection, but simply asserts this to be the case. This is improper.” (Response at pp. 9-10).
With regard to Bayesian optimization and Hyperband (BOHB) hyperparameter optimization, Applicant submits that “BOHB specifically requires fitting complex probabilistic models, such as Kernel Density Estimators (KDE), and calculating acquisition functions to decide where to sample next. These calculations are far beyond standard mental arithmetic, in particular because the unaided human mind is incapable of combining mentally a guided search in the form of Bayesian optimization that learns from past evaluations with a hyperband optimization that organizes the evaluations into a bracket-based structure that allows the BOHB to evaluate multiple configurations independently in a highly scalable way through the use of parallel computing resources.” (Response at p. 10 (emphasis added by Examiner)).
Also, Applicant submits under “Section 2107.07(a) of the MPEP, titled "Formulating a Rejection For Lack of Subject Matter Eligibility," states that "[w]hen making the rejection, the Office action must provide an explanation as to why each claim is unpatentable, which must be sufficiently clear and specific to provide applicant sufficient notice of the reasons for ineligibility and enable the applicant to effectively respond." (Response at p. 11).
Further, with relation to a “mathematical concept,” Applicant complains the “obligation to distinguish between a claim element that recites a mathematical concept and one that merely involves a mathematical concept is not addressed in the present Office Action.” (Response at p. 12).
Also, Applicant submits that “[f]or largely the same reasons [(b) determining an optimal parameterization is neither a mental process nor a mathematical concept], regarding the step of “’[(g)] optimizing the decision tree’ . . . recites neither a mental process nor a mathematical concept.” (Response at p. 13).
Examiner Response:
Examiner respectfully disagrees because, under Step 2A Prong One, the rejection above identifies the judicial exception (that is, abstract idea) by referring to what is recited in the claim and explain why it is considered an exception. For example, if the claim is directed to an abstract idea, the rejection should identify the abstract idea as it is recited in the claim and explain why it is an abstract idea. (MPEP § 2106.07(a)). Accordingly, Examiner’s rejections as set out above in detail comports with Office guidance.
Further, the plain meaning of the term “determining” is to find out or come to a decision about by investigation, reasoning, or calculation. In relation to “determining an optimal parameterization of the hyperparameters using BOHB,” this the use of a BOHB algorithm to determine optimal parameterization of the hyperparameters. The broadest reasonable interpretation is an activity that can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinions because the step involves making determinations and identifications, which are mental tasks humans routinely do, which is not inconsistent with the Applicant’s disclosure. (MPEP § 2111; see 2024 SME Guidelines, 89 Fed. Reg. 137, at p. 58136 (17 July 2024) (regarding “mental processes”)).
Similarly, the plain meaning of the term “optimizing” is improving for maximum efficiency or effectiveness. In relation to “optimizing the decision tree,” this activity is in relation to improving the functioning of the decision tree for outputting suitable parameterization. The broadest reasonable interpretation is an activity that can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and options directed to the decision tree because the step involves making determinations and identifications, which are mental tasks humans routinely do, which is not inconsistent with the Applicant’s disclosure. (MPEP § 2111; see 2024 SME Guidelines, 89 Fed. Reg. 137, at p. 58136 (17 July 2024) (regarding “mental processes”)).
With regard to Applicant’s argument that BOHB relies on “calculations [that] are far beyond standard mental arithmetic, in particular because the unaided human mind is incapable,” the use of a physical aid (e.g., pencil and paper or a slide rule) to help perform a mental step (e.g., a mathematical calculation) does not negate the mental nature of the limitation, but simply accounts for variations in memory capacity from one person to another. (MPEP § 2106.04(a)(2) sub III.B). Notably, a claim that requires a computer may still recite a mental process. (MPEP § 2106.04(a)(2) sub III.C).
Further, in view of that the BOHB relies on “calculations,” Examiner identifies the judicial exception as the abstract idea of a mathematical concept, (MPEP § 2106.04(a)(2) sub I), as set out above in detail.
Accordingly, under Step 2A Prong One, the rejection above identifies the judicial exception (that is, abstract idea) by referring to what is recited in the claim and explain why it is considered an exception.
17. Also, “determining step, page 3 of the Office Action splices it into two parts. First, the Patent Office isolates the word ‘determining’ in order to say that this word alone represents something that "can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinion, and accordingly, are a mental process, (MPEP § 2106.05(a)(2) sub Ill), which are one of the groupings of abstract ideas. . . . Applicant disagrees with this approach of splicing a single claim limitation into textual segments and applying distinct categories of abstract ideas to these segments. The Patent Office may consider that a claim limitation as a whole falls in more than one judicial exception category. MPEP 2106.04(a) (‘[S]ome claims recite limitations that/all within more than one grouping or sub-grouping. For example, a claim reciting performing mathematical calculations using a formula that could be practically performed in the human mind may be considered to fall within the mathematical concepts grouping and the mental process grouping.’). . . . Therefore, applying Prong One to individual words isolated from the overall claim limitation, which is what the Patent Office has done by applying Prong One separately to the word ‘determining,’ is improper.” (Response at pp. 8-9).
Examiner Response:
Examiner respectfully points out that the use of individual words, or textual segments, is for the purpose of referring to the claim limitation, and not intended to be the bases of the subject-matter evaluation under the Office guidance.
18. Applicant submits, in view of Desjardins, that “[t]o avoid this improper categorical exclusion of ‘AI innovations from patent protection,’ the Director admonished the PTAB and examining corps that they ‘should not evaluate claims at such a high level of generality.’ Yet that is precisely what the Patent Office has done in its Prong One analysis by using the analysis-free conclusory statement that the determining step is a ‘limitation[] that include[s] a mathematical concept, (MPEP § 2106.04(a)(2) sub I), which is one of the groupings of abstract ideas.’” (Response at p. 11).
Examiner Response:
Examiner respectfully submits that the rejection above identifies the judicial exception (that is, abstract idea) by referring to what is recited in the claim and explain why it is considered an exception. (MPEP § 2106.07(a)).
Also, the guidance of Desjardins provides that “[w]hen performing [the subject-matter] evaluation, examiners should be ‘careful to avoid oversimplifying the claims’ by looking at them generally and failing to account for the specific requirements of the claims.” (Advance Notice of Change to the MPEP in light of Ex Parte Desjardins, at p. 4 (05 December 2025)). The rejection the rejection above identifies the judicial exception (that is, abstract idea) by referring to what is recited in the claim and explain why it is considered an exception, because they can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinions. (MPEP § 2106.07(a); MPEP § 2106.04(a)(2) sub III). For example, the limitations of “[(b),(e))] determining,” “[(c)] assessing,” “[(d)] creating a matrix,” and “[(g)] optimizing” can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinion, and accordingly, are a mental process, (MPEP § 2106.05(f)(2) sub III), which are one of the groupings of abstract ideas.
Accordingly, the claims recite an abstract idea, as set out above in detail.
19. Under Step 2A Prong Two, Applicant refers to MPEP § 2106.04(d)(1) and Desjardin, Applicant submits that, “in evaluating the claims here, the Patent Office was obliged to consider not just the additional elements recited in the "determining" and "optimizing" alone, but in context with the rest of the claims.” (Response at pp. 14-15).
Applicant submits that per Section 2106.04(d)(1), the disclosure “results in concrete technological improvements in the form high-grade independent learning across all meta-training data sets, increased capacity to handle meta training data sets, and savings in computational resources and memory usage. Thus, these improvements are not improvements to a mere abstract idea; they carry palpable, measurable technological enhancements that redound to the benefit of machine learning systems. Moreover, with regard to the requirement in Section 2106.04(d)(1) that ‘the claim must be evaluated to ensure that the claim itself reflects the disclosed improvement,’ the above blurbs expressly link the above improvements to (1) the BOHB hyperparameter optimization of the determining step and (2) the decision tree optimization based on this hyperparameter optimization (via the claimed matrix) and on the meta-features." (Response at p. 16 (quoting Specification ¶¶ 0017, 0020, 0021)).
Examiner Response:
Under Step 2A Prong Two, the rejection identifies any additional elements recited in the claim beyond the identified judicial exception (i.e., abstract idea); and evaluate the integration of the judicial exception into a practical application by explaining that the claim as a whole, looking at the additional elements individually and in combination, does not integrate the judicial exception into a practical application using the considerations set forth in MPEP §§ 2106.04(d), 2106.05(a)- (c) and (e)- (h).
“Integration” may be based on the improvements in the functioning of a computer or an improvement to any other technology or technical field. (MPEP § 2106.04(d)(1)). The evaluation requires, [i]n sum, that (1) the specification should be evaluated to determine if the disclosure provides sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement. Next, (2) if the specification sets forth such an improvement, the claim must be evaluated to ensure that the claim itself reflects the disclosed improvement.
By way of example to Desjardins, the MPEP provides under Step 2A Prong Two that “the [Desjardins] specification identified improvements as to how the machine learning model itself operates, including training a machine learning model to learn new tasks while protecting knowledge about previous tasks to overcome the problem of ‘catastrophic forgetting’ encountered in continual learning systems. Importantly, the [appeals review panel (ARP)] evaluated the claims as a whole in discerning at least the limitation ‘adjust the first values of the plurality of parameters to optimize performance of the machine learning model on the second machine learning task while protecting performance of the machine learning model on the first machine learning task’ reflected the improvement disclosed in the specification. Accordingly, the claims as a whole integrated what would otherwise be a judicial exception instead into a practical application at Step 2A Prong Two, and therefore the claims were deemed to be outside any specific, enumerated judicial exception (Step 2A: NO).” (MPEP § 2106.04(d) sub III; see “Advance Notice of Change to the MPEP in light of Ex Parte Desjardins” (05 December 2025) at p. 2)).
In contrast to Desjardins, the Applicant points to language indicating an improvement, (Response at p. 15 (referring to Specification at p. 5, lines 6-16 (“The inventors have found that the combination of meta-learning and hyperparameter optimization results in high-grade domain-independent learning”))). In other words, the specification may explicitly sets forth an improvement but does so in a conclusory manner (i.e., a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art). Thus, the improvement appears directed to the abstract idea, which still is an abstract idea.
20. Regarding Step 2B, Applicant complains the Examiner’s “analysis fails to conform to the framework set forth in MPEP, Section 2106.05(d), for analyzing additional claim elements under Step 2B because the Patent Office merely offers conclusory statements baldly asserting that the claim elements in question ‘do not amount to significantly more than the abstract idea, instead of offering support in the form of evidence conforming to one or more of categories [of the Berkheimer criterium under Section 2106.07(a)](A)-(D).’” (Response at p. 17).
Examiner Response:
Examiner respectfully sets out that for Step 2B, the rejection explains why the additional elements, taken individually and in combination, do not result in the claim, as a whole, amounting to significantly more than the identified judicial exception. Further, certain claim elements recite well understood, routine, conventional activities in the relevant field, which is expressly supported in writing with one of the four options specified in Subsection III. (MPEP § 2106.07(a)).
The rejections above set out appropriate forms of support for the factual determinations under Step 2B. For example, the rejections under Step 2B either rely upon MPEP § 2106.05(f) relating to a generic computer component used to implement the abstract idea, or “[a] citation to one or more of the court decisions discussed in Subsection II below as noting the well-understood, routine, conventional nature of the additional element(s).” (MPEP § 2106.05(d) sub I.2).
For example, the additional elements recited in the claim beyond the identified judicial exception include a “computer-implemented method,” and a “system,” which are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)). Also, by way of example, the claim also recites, inter alia, “[(a)] providing predefined hyperparameters,” which is a well-understood, routine, and conventional activity of receiving and transmitting data over a network, (MPEP § 2106.05(d) sub II.i), that does not amount to significantly more than the abstract idea. Reference to Section 2106.05(d) sub II.i refers to subsection II.i, which recites authority for receiving or transmitting data over a network is a well-understood, routine, and conventional activity:
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added)).
Accordingly, the rejection explains why the additional elements, taken individually and in combination, do not result in the claim, as a whole, amounting to significantly more than the identified judicial exception. Further, certain claim elements recite well understood, routine, conventional activities in the relevant field, which is expressly supported in writing with one of the four options specified in Subsection III. (MPEP § 2106.07(a)).
Section 103
21. Applicant submits that “[u]sing a greedy algorithm to construct a set of policies is not the same as "wherein a subset of meta-features of the plurality of the meta-features is determined using a Greedy algorithm." Instead, although Feurer discloses that meta-features are used to select a particular policy ("we can use dataset meta-features to select, on a per-dataset basis, which policy to use"), it does not follow that using meta-features to select a policy that is part of a set of policies determined by a Greedy algorithm means that the meta-features are selected on the basis of a greedy algorithm.” (Response at p. 18).
Examiner Response:
Examiner notes that a Greedy algorithm is used to determine a subset of meta-features in the context of “optimizing the decision tree”:
* * *
[(g)] optimizing the decision tree as a function of the meta-features and of the matrix, . . . [(g.2)] wherein a subset of meta-features of the plurality of the meta-features is determined using a Greedy algorithm.
(claim 1, lines 19 & 21-22). In this regard, Examiner relies upon the teachings of Katz relating to the use of a greedy algorithm for meta-feature generation, as set out above in detail.
Conclusion
22. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
23. The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure:
(US Patent 10762163 to Fusi) teaches a probabilistic matrix factorization for automated machine learning to identify an optimum workflow to apply to a new data set for generating predicted output values from the new data set.
(Feurer et al., “Hyperparameter Optimization,” Springer (May 2019)) teaches an overview of the most prominent approaches for hyperparameter optimization (HPO). We first discuss blackbox function optimization methods based on model-free methods and Bayesian optimization. Since the high computational demand of many modern machine learning applications renders pure blackbox optimization extremely costly, we next focus on modern multi-fidelity methods that use (much) cheaper variants of the blackbox function to approximately assess the quality of hyperparameter settings.
24. Any inquiry concerning this communication or earlier communications from the Examiner should be directed to KEVIN L. SMITH whose telephone number is (571) 272-5964. Normally, the Examiner is available on Monday-Thursday 0730-1730.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the Examiner by telephone are unsuccessful, the Examiner’s supervisor, KAKALI CHAKI can be reached on 571-272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.L.S./
Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
1 Reference designations are used in the claims solely for the purpose of aiding in the evaluation for subject-matter eligibility.
Prosecution Insights