Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner’s Note
Providing supporting paragraph(s) for each limitation of amended/new claim(s) in Remarks is strongly requested for clear and definite claim interpretations by Examiner (e.g., to avoid rejections under 35 U.S.C § 112(a) “Lack of written description”)
Applicant can schedule interviews (via Automated Interview Request (AIR)) at any stage of the prosecution (e.g., Non-Final, Final, and After-Final) to discuss any issues related to, for example, rejections under 35 U.S.C § 101 and § 102/103, for moving toward allowance.
Priority
Acknowledgment is made of applicant's claim for the PCT application filed on 06/07/2021.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 3 is/are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim(s) 3 recite(s) the limitation “the number of decision rules” (line 4). There is insufficient antecedent basis for this limitation in the claim. It is not clear what it is referring to. It appears it may need to read “a number of decision rules”, or something else. For the purposes of examination, “a number of decision rules” is used.
Claim(s) 3 recite(s) the limitation “the number of conditions” (line 5). There is insufficient antecedent basis for this limitation in the claim. It is not clear what it is referring to. It appears that it needs to read “a number of conditions” or something else. For the purposes of examination, “a number of conditions” is used.
Claim(s) 3 each recite(s) limitations that raise issues of indefiniteness as set forth above, and their dependent claims are rejected at least based on their direct and/or indirect dependency from the claims listed above. Appropriate explanation and/or amendment is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1
Step 1: “Is the claim to a process, machine, manufacture, or composition of matter?”
The claim is directed to a system. Therefore, yes.
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?”
a prediction process of calculating a prediction result with use of predicted values of, among decision rules included in a decision list composed of the decision rules extracted from a decision rule set that is a set of decision rules each of which is a combination of a condition and a predicted value for a case where the condition is satisfied, K (K is a natural number of not less than 2) top-ranked decision rules whose conditions are satisfied by one of training examples included in a training example set; and (i.e., mathematical concept)
a list determining process of determining, from among a plurality of the decision lists generated from the decision rule set, a decision list to be output, the determining being made on a basis of a prediction result calculated for the training examples included in the training example set. (i.e., mental process)
The claim is directed to an abstract idea. Therefore, yes.
Step 2A Prong 2: “Does the claim recite additional elements that integrate the judicial exception into a practical application?” The following elements are directed to additional elements:
An information processing apparatus: including at least one processor, the at least one processor executing: (well-understood, routine, and conventional generic computer and/or model, see MPEP 2106.05(f))
Therefore, no.
Step 2B: “Does the claim recite additional elements that amount to significantly more than the judicial exception?”
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Specifically, the claimed inventions simply append well-understood, routine and conventional activities previously known to the industry, both when viewed independently and as an ordered combination, specified at a high level of generality, to the judicial exception, (e.g., a claim to an abstract idea requiring no more than a generic computer to perform generic computer functions that are well-understood, routine and conventional activities previously known to the industry). Therefore, no.
Regarding claim 2
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?”
the prediction process calculates, with use of the decision list represented by a variable indicative of a position in the decision list at which position a decision rule included in the decision rule set is located, a value of an objective function including an error term indicative of an error of the prediction result; and (i.e., mathematical concept)
the list determining process determines the decision list to be output by repeatedly carrying out a process of updating the variable on a basis of the calculated value of the objective function until the value of the objective function satisfies a predetermined condition. (i.e., mental process)
The claim is directed to an abstract idea. Therefore, yes.
The claim does not add any additional elements (Step 2A Prong 2) or significantly more (Step 2B).
Regarding claim 3
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?”
the prediction process calculates the value of the objective function including (i) a constraint term relating to the number of decision rules included in the decision list or (ii) a constraint term relating to the number of conditions included in the decision rules included in the decision list. (i.e., mathematical concept)
The claim is directed to an abstract idea. Therefore, yes.
The claim does not add any additional elements (Step 2A Prong 2) or significantly more (Step 2B).
Regarding claim 4
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?” The claim recites the abstract idea identified above regarding claim 2. Therefore, yes.
Step 2A Prong 2: “Does the claim recite additional elements that integrate the judicial exception into a practical application?” The following elements are directed to additional elements:
the variable includes, for each of the training examples included in the training example set, variables indicative of the K decision rules which are a first to K-th decision rules in the decision list and whose conditions are satisfied by one of the training examples (a particular type or source of model/data, Field of Use and Technological Environment, see MPEP 2106.05(h))
Therefore, no.
Step 2B: “Does the claim recite additional elements that amount to significantly more than the judicial exception?”
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Therefore, no.
Regarding claim 5
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?”
the prediction process calculates the prediction result with use of the value of the K, the value having been accepted by the acceptance process. (i.e., mathematical concept)
The claim is directed to an abstract idea. Therefore, yes.
Step 2A Prong 2: “Does the claim recite additional elements that integrate the judicial exception into a practical application?” The following elements are directed to additional elements:
an acceptance process of accepting setting of a value of the K, wherein (insignificant extra-solution activity of mere data gathering, see MPEP 2106.05(g))
Therefore, no.
Step 2B: “Does the claim recite additional elements that amount to significantly more than the judicial exception?”
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Therefore, no.
Regarding claim 6
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?”
a decision rule set generating process of (a) generating a decision rule by extracting, from at least one decision tree included in a decision tree set including the at least one decision tree, each condition appearing on a path from a root to a leaf of the at least one decision tree and (b) generating the decision rule set including the generated decision rule. (i.e., mental process)
The claim is directed to an abstract idea. Therefore, yes.
The claim does not add any additional elements (Step 2A Prong 2) or significantly more (Step 2B).
Regarding claim 7
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?”
a rank setting process of ranking the decision rules included in the decision rule set, (i.e., mental process)
wherein the prediction process calculates the prediction result with use of the K top- ranked predicted values (i.e., mathematical concept)
The claim is directed to an abstract idea. Therefore, yes.
Therefore, no.
The claim does not add any additional elements (Step 2A Prong 2) or significantly more (Step 2B).
Regarding claim 8
Step 1: “Is the claim to a process, machine, manufacture, or composition of matter?”
The claim is directed to a composition of matter. Therefore, yes.
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?”
the rank setting process ranks the decision rules on a basis of differences between the predicted values for the training examples satisfying conditions of the decision rules and a predicted value to be compared (i.e., mental process)
The claim is directed to an abstract idea. Therefore, yes.
The claim does not add any additional elements (Step 2A Prong 2) or significantly more (Step 2B).
Regarding claim 9
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?”
a prediction process of calculating a prediction result with use of predicted values of, among decision rules included in a decision list composed of the decision rules each of which is a combination of a condition and a predicted value for a case where the condition is satisfied, K (K is a natural number of not less than 2) top-ranked decision rules whose conditions are satisfied by the input data (i.e., mathematical concept)
The claim is directed to an abstract idea. Therefore, yes.
Step 2A Prong 2: “Does the claim recite additional elements that integrate the judicial exception into a practical application?” The following elements are directed to additional elements:
An information processing apparatus including at least one processor, the at least one processor executing: (well-understood, routine, and conventional generic computer and/or model, see MPEP 2106.05(f))
an input data acquiring process of acquiring input data to be subjected to prediction; and (insignificant extra-solution activity of mere data gathering, see MPEP 2106.05(g))
Therefore, no.
Step 2B: “Does the claim recite additional elements that amount to significantly more than the judicial exception?”
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Therefore, no.
Regarding claim 10
The claim is rejected for the reasons set forth in the rejection of Claim 1 under 35 U.S.C. 101, mutatis mutandis.
Regarding claim 11
The claim is rejected for the reasons set forth in the rejection of Claim 1 under 35 U.S.C. 101, mutatis mutandis.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rudin et al. (Learning customized and optimized lists of rules with mathematical programming) in view of FRIEDMAN et al. (PREDICTIVE LEARNING VIA RULE ENSEMBLES)
Regarding claim 1
Rudin teaches
An information processing apparatus: including at least one processor, the at least one processor executing:
(Rudin [sec(s) 6] “For all baseline methods, we used the Weka (Hall et al, 2009) software package on a Mac-Book Pro with 2.53 GHz Intel i5 core processor. Several of the algorithms have no parameters to tune (AdaBoost, logistic regression). Heuristics were used to set some of the parameters for SVM (C=1, RBF kernel parameter = 1/#Features), though we report results from SVM for many possible parameter values for all datasets in Table 11 in Appendix A. Default parameters were used for random forests.”;)
a prediction process of calculating a prediction result with use of predicted values of,
(Rudin [sec(s) 1] “The selection and order of these rules determines accuracy. The fractions reported between parentheses are empirical conditional probabilities of making a correct prediction for a given rule.”;)
among decision rules included in a decision list composed of the decision rules extracted from a decision rule set that is a set of decision rules each of which is a combination of a condition and a predicted value for a case where the condition is satisfied,
(Rudin [sec(s) 1] “Our method for producing rule lists works in two steps, like an associative classification technique: first, in Step 1, a set of “IF-THEN” association rules with high support and high confidence are mined using a rule-mining algorithm, and second, in Step 2 we learn the ordering of the rules from data to form a sparse and accurate decision list. We use discrete optimization techniques that are designed to handle the combinatorial explosion caused by a combinatorially increasing number of possible rules, and an exponentially increasing number of ways to order them. We provide two mathematical programming formulations: one for mining the rules, and the other for ordering them. Each of the two formulations can be used independently; for instance it is possible to use our formulation for learning the ordering of rules when the rules themselves were generated from a separate rule mining algorithm. Alternatively, it is possible to use our rule mining algorithm for examining rules that might be individually interesting. Both are mixed-integer linear programs: constraints and the objective are linear in the decision variables.” [sec(s) 2] “Their structural model takes the form (3)
PNG
media_image1.png
164
598
media_image1.png
Greyscale
, where M is the size of the ensemble and each ensemble member (“base learner”) fm(x) is a different function of the input variables x derived from the training data.” [sec(s) 3] “Rule based ensembles. The base learners considered here are simple rules. Let Sj be the set of all possible values for input variable xj, xj ∈ Sj, and sjm be a specified subset of those values, sjm⊆ Sj. Then each base learner takes the form of a conjunctive rule (6)
PNG
media_image2.png
167
556
media_image2.png
Greyscale
, where I(·) is an indicator of the truth of its argument.”;)
K (K is a natural number [of not less than 2]) top-ranked decision rule[s] whose conditions are satisfied by one of training examples included in a training example set; and
(Rudin [sec(s) 1] “Our goal is to produce non-black-box machine learning models comprised of logical IF-THEN rule statements. We want these models to be easily customizable by the user, and to have training performance that is optimized. … The selection and order of these rules determines accuracy. The fractions reported between parentheses are empirical conditional probabilities of making a correct prediction for a given rule. For instance, the empirical conditional probability 30/35 for the second rule means that 35 observations did not satisfy the first rule, but satisfied the second rule. Of those, in 30 observations the emergency room was at capacity. The order of rules is important because in a given situation, the rule that makes the prediction is the first rule that applies to it.” [sec(s) Abs] “We introduce a mathematical programming approach to building rule lists, which are a type of interpretable, nonlinear, and logical machine learning classifier involving IF-THEN rules.” [sec(s) 1] “Our method for producing rule lists works in two steps, like an associative classification technique: first, in Step 1, a set of “IF-THEN” association rules with high support and high confidence are mined using a rule-mining algorithm, and second, in Step 2 we learn the ordering of the rules from data to form a sparse and accurate decision list. … We provide two mathematical programming formulations: one for mining the rules, and the other for ordering them. Each of the two formulations can be used independently; for instance it is possible to use our formulation for learning the ordering of rules when the rules themselves were generated from a separate rule mining algorithm. Alternatively, it is possible to use our rule mining algorithm for examining rules that might be individually interesting. Both are mixed-integer linear programs: constraints and the objective are linear in the decision variables.”;)
a list determining process of determining, from among a plurality of the decision lists generated from the decision rule set, a decision list to be output, the determining being made on a basis of a prediction result calculated for the training examples included in the training example set.
(Rudin [sec(s) Abs] “We introduce a mathematical programming approach to building rule lists, which are a type of interpretable, nonlinear, and logical machine learning classifier involving IF-THEN rules. Unlike traditional decision tree algorithms like CART and C5.0, this method does not use greedy splitting and pruning. Instead, it aims to fully optimize a combination of accuracy and sparsity, obeying user-defined constraints. This method is useful for producing non-black-box predictive models, and has the benefit of a clear user-defined tradeoff between training accuracy and sparsity.” [sec(s) 1] “Our method for producing rule lists works in two steps, like an associative classification technique: first, in Step 1, a set of “IF-THEN” association rules with high support and high confidence are mined using a rule-mining algorithm, and second, in Step 2 we learn the ordering of the rules from data to form a sparse and accurate decision list. We use discrete optimization techniques that are designed to handle the combinatorial explosion caused by a combinatorially increasing number of possible rules, and an exponentially increasing number of ways to order them. We provide two mathematical programming formulations: one for mining the rules, and the other for ordering them. Each of the two formulations can be used independently; for instance it is possible to use our formulation for learning the ordering of rules when the rules themselves were generated from a separate rule mining algorithm. Alternatively, it is possible to use our rule mining algorithm for examining rules that might be individually interesting. Both are mixed-integer linear programs: constraints and the objective are linear in the decision variables.”;)
However, the combination of Rudin does not appear to explicitly teach:
K (K is a natural number [of not less than 2]) top-ranked decision rule[s] whose conditions are satisfied by one of training examples included in a training example set; and
FRIEDMAN teaches
K (K is a natural number of not less than 2) top-ranked decision rules whose conditions are satisfied by one of training examples included in a training example set; and
(FRIEDMAN [sec(s) Abs] “General regression and classification models are constructed as linear combinations of simple rules derived from the data. Each rule consists of a conjunction of a small number of simple statements concerning the values of individual input variables.” [sec(s) 2] “Ensemble learning. Learning ensembles have emerged as being among the most powerful learning methods [see Breiman (1996, 2001), Freund and Schapire (1996), Friedman (2001)]. Their structural model takes the form (3)
PNG
media_image1.png
164
598
media_image1.png
Greyscale
, where M is the size of the ensemble and each ensemble member (“base learner”) fm(x) is a different function of the input variables x derived from the training data. Ensemble predictions F(x) are taken to be a linear combination of the predictions.” [sec(s) 3] “Rule based ensembles. The base learners considered here are simple rules. Let Sj be the set of all possible values for input variable xj, xj ∈ Sj, and sjm be a specified subset of those values, sjm⊆ Sj. Then each base learner takes the form of a conjunctive rule (6)
PNG
media_image2.png
167
556
media_image2.png
Greyscale
, where I(·) is an indicator of the truth of its argument. Each such base learner assumes two values rm(x) ∈{0,1}. It is nonzero when all of the input variables realize values that are simultaneously within their respective subsets {xj ∈ sjm}n1.”;)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Rudin with the top-ranked decision rules of FRIEDMAN.
One of ordinary skill in the art would have been motived to combine in order to provide a accuracy which is competitive with the best tree based ensembles, and automatically identify the variables that are involved in interactions with other variables, the strength and degree of those interactions, as well as the identities of the other variables with which they interact.
(FRIEDMAN [sec(s) Abs] “Techniques are presented for automatically identifying those variables that are involved in interactions with other variables, the strength and degree of those interactions, as well as the identities of the other variables with which they interact. Graphical representations are used to visualize both main and interaction effects” [sec(s) 4] “The results presented in Figure 2 suggest that the rule based approach to ensem ble learning described in Section 3 produces accuracy comparable to that based on decision trees. Other tree based ensemble methods including bagging and ran dom forests were compared to those presented here (MART, ISLE) in Friedman andPopescu(2003), and seen to exhibit somewhat lower accuracy over these 100 regression and classification data sets. Thus, rule based ensembles appear to be competitive in accuracy with the best tree based ensembles.”)
Regarding claim 2
The combination of Rudin, FRIEDMAN teaches claim 1.
Rudin further teaches
the prediction process calculates, with use of the decision list represented by a variable indicative of a position in the decision list at which position a decision rule included in the decision rule set is located, a value of an objective function including an error term indicative of an error of the prediction result; and
(Rudin [sec(s) Abs] “We introduce a mathematical programming approach to building rule lists, which are a type of interpretable, nonlinear, and logical machine learning classifier involving IF-THEN rules. Unlike traditional decision tree algorithms like CART and C5.0, this method does not use greedy splitting and pruning. Instead, it aims to fully optimize a combination of accuracy and sparsity, obeying user-defined constraints. This method is useful for producing non-black-box predictive models, and has the benefit of a clear user-defined tradeoff between training accuracy and sparsity.” [sec(s) 1] “Our method for producing rule lists works in two steps, like an associative classification technique: first, in Step 1, a set of “IF-THEN” association rules with high support and high confidence are mined using a rule-mining algorithm, and second, in Step 2 we learn the ordering of the rules from data to form a sparse and accurate decision list. We use discrete optimization techniques that are designed to handle the combinatorial explosion caused by a combinatorially increasing number of possible rules, and an exponentially increasing number of ways to order them. We provide two mathematical programming formulations: one for mining the rules, and the other for ordering them. Each of the two formulations can be used independently; for instance it is possible to use our formulation for learning the ordering of rules when the rules themselves were generated from a separate rule mining algorithm. Alternatively, it is possible to use our rule mining algorithm for examining rules that might be individually interesting. Both are mixed-integer linear programs: constraints and the objective are linear in the decision variables.”;)
the list determining process determines the decision list to be output by repeatedly carrying out a process of updating the variable on a basis of the calculated value of the objective function until the value of the objective function satisfies a predetermined condition.
(Rudin [sec(s) Abs] “We introduce a mathematical programming approach to building rule lists, which are a type of interpretable, nonlinear, and logical machine learning classifier involving IF-THEN rules. Unlike traditional decision tree algorithms like CART and C5.0, this method does not use greedy splitting and pruning. Instead, it aims to fully optimize a combination of accuracy and sparsity, obeying user-defined constraints. This method is useful for producing non-black-box predictive models, and has the benefit of a clear user-defined tradeoff between training accuracy and sparsity.” [sec(s) 1] “Our method for producing rule lists works in two steps, like an associative classification technique: first, in Step 1, a set of “IF-THEN” association rules with high support and high confidence are mined using a rule-mining algorithm, and second, in Step 2 we learn the ordering of the rules from data to form a sparse and accurate decision list. We use discrete optimization techniques that are designed to handle the combinatorial explosion caused by a combinatorially increasing number of possible rules, and an exponentially increasing number of ways to order them. We provide two mathematical programming formulations: one for mining the rules, and the other for ordering them. Each of the two formulations can be used independently; for instance it is possible to use our formulation for learning the ordering of rules when the rules themselves were generated from a separate rule mining algorithm. Alternatively, it is possible to use our rule mining algorithm for examining rules that might be individually interesting. Both are mixed-integer linear programs: constraints and the objective are linear in the decision variables.”;)
Regarding claim 3
The combination of Rudin, FRIEDMAN teaches claim 2.
Rudin further teaches
the prediction process calculates the value of the objective function including (i) a constraint term relating to the number of decision rules included in the decision list or (ii) a constraint term relating to the number of conditions included in the decision rules included in the decision list.
(Rudin [sec(s) Abs] “We introduce a mathematical programming approach to building rule lists, which are a type of interpretable, nonlinear, and logical machine learning classifier involving IF-THEN rules. Unlike traditional decision tree algorithms like CART and C5.0, this method does not use greedy splitting and pruning. Instead, it aims to fully optimize a combination of accuracy and sparsity, obeying user-defined constraints. This method is useful for producing non-black-box predictive models, and has the benefit of a clear user-defined tradeoff between training accuracy and sparsity.” [sec(s) 1] “Our method for producing rule lists works in two steps, like an associative classification technique: first, in Step 1, a set of “IF-THEN” association rules with high support and high confidence are mined using a rule-mining algorithm, and second, in Step 2 we learn the ordering of the rules from data to form a sparse and accurate decision list. We use discrete optimization techniques that are designed to handle the combinatorial explosion caused by a combinatorially increasing number of possible rules, and an exponentially increasing number of ways to order them. We provide two mathematical programming formulations: one for mining the rules, and the other for ordering them. Each of the two formulations can be used independently; for instance it is possible to use our formulation for learning the ordering of rules when the rules themselves were generated from a separate rule mining algorithm. Alternatively, it is possible to use our rule mining algorithm for examining rules that might be individually interesting. Both are mixed-integer linear programs: constraints and the objective are linear in the decision variables.”;)
Regarding claim 4
The combination of Rudin, FRIEDMAN teaches claim 2.
FRIEDMAN further teaches
the variable includes, for each of the training examples included in the training example set, variables indicative of the K decision rules which are a first to K-th decision rules in the decision list and whose conditions are satisfied by one of the training examples.
(FRIEDMAN [sec(s) Abs] “General regression and classification models are constructed as linear combinations of simple rules derived from the data. Each rule consists of a conjunction of a small number of simple statements concerning the values of individual input variables.” [sec(s) 2] “Ensemble learning. Learning ensembles have emerged as being among the most powerful learning methods [see Breiman (1996, 2001), Freund and Schapire (1996), Friedman (2001)]. Their structural model takes the form (3)
PNG
media_image1.png
164
598
media_image1.png
Greyscale
, where M is the size of the ensemble and each ensemble member (“base learner”) fm(x) is a different function of the input variables x derived from the training data. Ensemble predictions F(x) are taken to be a linear combination of the predictions.” [sec(s) 3] “Rule based ensembles. The base learners considered here are simple rules. Let Sj be the set of all possible values for input variable xj, xj ∈ Sj, and sjm be a specified subset of those values, sjm⊆ Sj. Then each base learner takes the form of a conjunctive rule (6)
PNG
media_image2.png
167
556
media_image2.png
Greyscale
, where I(·) is an indicator of the truth of its argument. Each such base learner assumes two values rm(x) ∈{0,1}. It is nonzero when all of the input variables realize values that are simultaneously within their respective subsets {xj ∈ sjm}n1.”;)
The combination of Rudin, FRIEDMAN is combinable with FRIEDMAN for the same rationale as set forth above with respect to claim 1.
Regarding claim 5
The combination of Rudin, FRIEDMAN teaches claim 1.
Rudin further teaches
an acceptance process of accepting setting of a value of the K, wherein
(Rudin [sec(s) Abs] “We introduce a mathematical programming approach to building rule lists, which are a type of interpretable, nonlinear, and logical machine learning classifier involving IF-THEN rules. Unlike traditional decision tree algorithms like CART and C5.0, this method does not use greedy splitting and pruning. Instead, it aims to fully optimize a combination of accuracy and sparsity, obeying user-defined constraints. This method is useful for producing non-black-box predictive models, and has the benefit of a clear user-defined tradeoff between training accuracy and sparsity.” [sec(s) 2] “These issues illustrate where mathematical programming has a clear advantage; it allows us to fully optimize the selection and order of rules to be accurate, sparse, and obey user-defined constraints. There is a tradeoff: classical methods are very fast but not very accurate, and work on large datasets, but it can be difficult to customize them to obey user-defined constraints.”; Note that FRIEDMAN teaches “K”.)
FRIEDMAN further teaches
the prediction process calculates the prediction result with use of the value of the K, the value having been accepted by the acceptance process.
(FRIEDMAN [sec(s) Abs] “General regression and classification models are constructed as linear combinations of simple rules derived from the data. Each rule consists of a conjunction of a small number of simple statements concerning the values of individual input variables.” [sec(s) 2] “Ensemble learning. Learning ensembles have emerged as being among the most powerful learning methods [see Breiman (1996, 2001), Freund and Schapire (1996), Friedman (2001)]. Their structural model takes the form (3)
PNG
media_image1.png
164
598
media_image1.png
Greyscale
, where M is the size of the ensemble and each ensemble member (“base learner”) fm(x) is a different function of the input variables x derived from the training data. Ensemble predictions F(x) are taken to be a linear combination of the predictions.” [sec(s) 3] “Rule based ensembles. The base learners considered here are simple rules. Let Sj be the set of all possible values for input variable xj, xj ∈ Sj, and sjm be a specified subset of those values, sjm⊆ Sj. Then each base learner takes the form of a conjunctive rule (6)
PNG
media_image2.png
167
556
media_image2.png
Greyscale
, where I(·) is an indicator of the truth of its argument. Each such base learner assumes two values rm(x) ∈{0,1}. It is nonzero when all of the input variables realize values that are simultaneously within their respective subsets {xj ∈ sjm}n1.”;)
The combination of Rudin, FRIEDMAN is combinable with FRIEDMAN for the same rationale as set forth above with respect to claim 1.
Regarding claim 6
The combination of Rudin, FRIEDMAN teaches claim 1.
FRIEDMAN further teaches
a decision rule set generating process of (a) generating a decision rule by extracting, from at least one decision tree included in a decision tree set including the at least one decision tree, each condition appearing on a path from a root to a leaf of the at least one decision tree and (b) generating the decision rule set including the generated decision rule.
(FRIEDMAN [sec(s) 3.1] “The approach used here is to view a decision tree as defining a collection of rules and take advantage of existing fast algorithms for producing decision tree ensembles. That is, decision trees are used as the base learner f(x;p) in Algorithm 1. Each node (interior and terminal) of each resulting tree fm(x) produces a rule of the form (7).” [sec(s) 3] “Rule based ensembles. The base learners considered here are simple rules. Let Sj be the set of all possible values for input variable xj, xj ∈ Sj, and sjm be a specified subset of those values, sjm⊆ Sj. Then each base learner takes the form of a conjunctive rule (6)
PNG
media_image2.png
167
556
media_image2.png
Greyscale
, where I(·) is an indicator of the truth of its argument. Each such base learner assumes two values rm(x) ∈{0,1}. It is nonzero when all of the input variables realize values that are simultaneously within their respective subsets {xj ∈ sjm}n1.” [sec(s) 3.2] “The total number of rules is (8)
PNG
media_image3.png
190
490
media_image3.png
Greyscale
, where tm is the number of terminal nodes for the mth tree” [sec(s) 2] “Ensemble learning. Learning ensembles have emerged as being among the most powerful learning methods [see Breiman (1996, 2001), Freund and Schapire (1996), Friedman (2001)]. Their structural model takes the form (3)
PNG
media_image1.png
164
598
media_image1.png
Greyscale
, where M is the size of the ensemble and each ensemble member (“base learner”) fm(x) is a different function of the input variables x derived from the training data. Ensemble predictions F(x) are taken to be a linear combination of the predictions.” [sec(s) Abs] “General regression and classification models are constructed as linear combinations of simple rules derived from the data. Each rule consists of a conjunction of a small number of simple statements concerning the values of individual input variables. These rule ensembles are shown to produce predictive accuracy comparable to the best methods.”;)
The combination of Rudin, FRIEDMAN is combinable with FRIEDMAN for the same rationale as set forth above with respect to claim 1.
Regarding claim 7
The combination of Rudin, FRIEDMAN teaches claim 1.
FRIEDMAN further teaches
a rank setting process of ranking the decision rules included in the decision rule set,
(FRIEDMAN [sec(s) 6] “A commonly used measure of relevance or importance Ik of any predictor in a linear model such as (25) is the absolute value of the coefficient of the corresponding standardized predictor. For rules, this becomes (28)
PNG
media_image4.png
84
659
media_image4.png
Greyscale
, where sk is the rule support (12). … The importance measures (28) and (29) are global in that they reflect the average influence of each predictor over the distribution of all joint input variable values. A corresponding local measure of influence at each point x in that space is for rules (7)” [sec(s) 2] “Ensemble learning. Learning ensembles have emerged as being among the most powerful learning methods [see Breiman (1996, 2001), Freund and Schapire (1996), Friedman (2001)]. Their structural model takes the form (3)
PNG
media_image1.png
164
598
media_image1.png
Greyscale
, where M is the size of the ensemble and each ensemble member (“base learner”) fm(x) is a different function of the input variables x derived from the training data. Ensemble predictions F(x) are taken to be a linear combination of the predictions.”;)
wherein the prediction process calculates the prediction result with use of the K top-ranked predicted values.
(FRIEDMAN [sec(s) 3.1] “The approach used here is to view a decision tree as defining a collection of rules and take advantage of existing fast algorithms for producing decision tree ensembles. That is, decision trees are used as the base learner f(x;p) in Algorithm 1. Each node (interior and terminal) of each resulting tree fm(x) produces a rule of the form (7).” [sec(s) 2] “Ensemble learning. Learning ensembles have emerged as being among the most powerful learning methods [see Breiman (1996, 2001), Freund and Schapire (1996), Friedman (2001)]. Their structural model takes the form (3)
PNG
media_image1.png
164
598
media_image1.png
Greyscale
, where M is the size of the ensemble and each ensemble member (“base learner”) fm(x) is a different function of the input variables x derived from the training data. Ensemble predictions F(x) are taken to be a linear combination of the predictions.” [sec(s) Abs] “General regression and classification models are constructed as linear combinations of simple rules derived from the data. Each rule consists of a conjunction of a small number of simple statements concerning the values of individual input variables. These rule ensembles are shown to produce predictive accuracy comparable to the best methods.”;)
The combination of Rudin, FRIEDMAN is combinable with FRIEDMAN for the same rationale as set forth above with respect to claim 1.
Regarding claim 8
The combination of Rudin, FRIEDMAN teaches claim 7.
FRIEDMAN further teaches
the rank setting means process ranks the decision rules on a basis of differences between the predicted values for the training examples satisfying conditions of the decision rules and a predicted value to be compared.
(FRIEDMAN [sec(s) 3] “Rule based ensembles. The base learners considered here are simple rules. Let Sj be the set of all possible values for input variable xj, xj ∈ Sj, and sjm be a specified subset of those values, sjm⊆ Sj. Then each base learner takes the form of a conjunctive rule (6)
PNG
media_image2.png
167
556
media_image2.png
Greyscale
, where I(·) is an indicator of the truth of its argument. Each such base learner assumes two values rm(x) ∈{0,1}. It is nonzero when all of the input variables realize values that are simultaneously within their respective subsets {xj ∈ sjm}n1.” [sec(s) 6] “sk is the rule support (12). … The importance measures (28) and (29) are global in that they reflect the average influence of each predictor over the distribution of all joint input variable values. A corresponding local measure of influence at each point x in that space is for rules (7) (30)
PNG
media_image5.png
65
650
media_image5.png
Greyscale
” [sec(s) 2] “Ensemble learning. Learning ensembles have emerged as being among the most powerful learning methods [see Breiman (1996, 2001), Freund and Schapire (1996), Friedman (2001)]. Their structural model takes the form (3)
PNG
media_image1.png
164
598
media_image1.png
Greyscale
, where M is the size of the ensemble and each ensemble member (“base learner”) fm(x) is a different function of the input variables x derived from the training data. Ensemble predictions F(x) are taken to be a linear combination of the predictions.”;)
The combination of Rudin, FRIEDMAN is combinable with FRIEDMAN for the same rationale as set forth above with respect to claim 1.
Regarding claim 9
The claim is rejected for the reasons set forth in the rejection of Claim 1.
In addition, Rudin further teaches
an input data acquiring process of acquiring input data to be subjected to prediction;
(Rudin [sec(s) 3] “The input to the rule-ordering algorithm includes a training set of labeled observations {(xi,yi)}mi=1, with observations xi ∈ X, and labels yi ∈ {−1,1} (in the example, “at capacity” might be assigned the label +1). The method here can be generalized to the multi-class setting easily. Also given as input is a collection of rules, which comes from a rule mining algorithm, or if the problem is small enough, we can input all possible rules below a certain size. R is the total number of rules available. Starting from the top of the list, each rule r evaluates each observation xi to determine whether it applies, meaning that the “IF” condition is met, and if so, the rule assigns a predicted label of either-1 or 1. We define an m×Rmatrix of parameters A, called the applies matrix.”;)
Regarding claim 10
The claim is rejected for the reasons set forth in the rejection of Claim 1.
Regarding claim 11
The combination of Rudin, FRIEDMAN teaches claim 1.
In addition, the claim is rejected for the reasons set forth in the rejection of Claim 1.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEHWAN KIM whose telephone number is (571)270-7409. The examiner can normally be reached Mon - Fri 9:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J Huntley can be reached on (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SEHWAN KIM/Examiner, Art Unit 2129
Prosecution Insights