Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is responsive to the Application filed on 01/02/2026
Claims 1-20 are pending in this case. Claims 1, 11 and 20 are independent claims. Claims 1, 11 and 20 have been currently amended.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 01/02/2026 has been entered.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 6-8, 10-14, 16-18 and 20 are rejected under 35 U.S.C. 103 as beingUnpatentable over “Distilling a Neural Network into a soft decision tree".https://arxiv.org/pdf/1711.09784, Frost et al., 11/27/2017, hereinafter referred to as Frosst in view of “Multi-Level Deep Cascade Trees for Conversion Rate Prediction in Recommendation System”, https://arxiv.org/pdf/1805.09484, Hong et al., 11/19/2018, hereinafter referred to as Hong.
With respect to claim 1, Frosst disclose:
A system for automatic conversion of a target machine learning model into differentiable cascading decision tree (CDT)data structure adapted for improved machine learning explainability, the system comprising: a computer processor operating in conjunction with computer memory and a non-transitory computer readable medium, the computer processor configured to (On page 2, The Hierarchical Mixture of Bigots, Frosst discloses a deep neural net into a differentiable decision tree.)
Instantiate the cascading differentiable tree data structure having at least one feature learning tree data structure and a decision making tree data structure, the cascading differentiable tree data structure [[is]] capable of generating an output distribution data structure based on processing a set of data values representing a function of raw features x (On page 2, The Hierarchical Mixture of Bigots, Frosst discloses a differentiable decision tree, where each inner node has a learned filter and a bias, and each leaf node has a learned distribution. Fig. 1 & page 3 disclose a differentiable decision tree that can produce an output distribution based on learnable features f representing a corresponding
With respect to claim 1, Frosst does not explicitly disclose:
function sigma (wx+b). Fig. 2 & page 5 further disclose the cascading order of a soft decision tree.)
Iteratively train, until convergence. the cascading differentiable tree data structure against the target machine learning model to mimic representations of the target machine learning model: wherein each of the at least one feature learning tree data structure includes one or more decision nodes that represent the function #(x; w) of raw features x given a set of parameters w and one or more leaf nodes that each represent an intermediate learned feature f based at least on a corresponding intermediate feature representation function f =f(x;w) (In Fig. 1 and page 3, Frosst discloses that each of the at least one feature learning tree data structures includes one or more decision nodes that represent the function sigma(wx+b) of the learnable feature. The soft decision tree is iteratively training the targeted machine learning model to mimic representations using a loss function that seeks to minimize the cross entropy between each leaf, weighted by its path probability, and the target distribution.
Wherein the decision making tree data structure includes one or more decision nodes each representing a corresponding function 0(f; w') of intermediate learned features f given parameters w' from the feature representation functions of the feature learning tree data structure (Fig. 1 & page 3, Frosst disclose a decision tree including one or more nodes each representing a corresponding function sigma(wx+b) of the learnable features given output distribution of Ql or Qr.)
Wherein the at least one feature learning tree data structure is coupled in a cascaded arrangement with the decision making tree data structure (In Fig.2 , Frosst disclose the feature learning tree is connected in a series with the soft decision tree)
With respect to claim 1, Frosst does not explicitly disclose:
Wherein the at least one feature learning tree data structure is coupled in a cascaded arrangement with the decision making tree data structure, and wherein there are two or more feature learning tree data structures that are coupled in the cascaded arrangement such that an intermediate learned feature f of a preceding feature learning tree data structures is provided as a raw input into a subsequent feature learning tree data structure. and the intermediate learned feature f of a final feature learning tree data structure is then provided to the decision making tree data structure
However, it is known by Hong to disclose:
Wherein the at least one feature learning tree data structure is coupled in a cascaded arrangement with the decision making tree data structure, and wherein there are two or more feature learning tree data structures that are coupled in the cascaded arrangement such that an intermediate learned feature f of a preceding feature learning tree data structures is provided as a raw input into a subsequent feature learning tree data structure. and the intermediate learned feature f of a final feature learning tree data structure is then provided to the decision making tree data structure (Fig. 2 & On page 1-2 (Tree Based Feature Representation), Hong disclose Multiple GBDT (Gradient Boosted Decision Tree), outputs/features learned by an earlier tree level are fed into a subsequent tree level. The final cascade output is used for prediction.
Frosst, in view of Hong, they are analogous pieces of art because both references concern Gradient Boosting Decision Trees, which are decision tree ensemble methods exploiting deep cascade structures and using a cross-entropy-based feature representation. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to introduce predictions of a complex deep neural network into a single, easy-to-understand decision tree as taught by Frosst, with multidimensional representational feature vector from preceding level and outputs its processing results to the next level by employing Gradient Boosting Decision Trees (GBDT) models as taught by Hong. The motivation for doing so would have been to improve the interpretability of deep neural networks (CNNs) and the generalization of standard decision trees of Frosst.
Regarding claim 2, Frosst in view of Hong disclose the elements of claim 1. In addition, Frosst disclose:
The system of claim 1, wherein the cascading differentiable tree data structure is adapted to distill the target machine learning architecture by training the cascading differentiable tree data structure based on the target machine learning architecture to mimic an input output function of the target machine learning architecture (Frosst disclose distilling a neural network into a soft decision tree.)
Regarding claim 3, Frosst in view of Hong disclose the elements of claim 1. In addition, Frosst disclose:
The method of claim 1, wherein depth values of the two or more feature learning tree data structures is a tunable parameter to adjust a balance between the two or more feature learning tree data structures (On page 4, first paragraph, Frosst disclose the size of the tree and then using mini-batch gradient descent to update all of their parameters simultaneously.)
Regarding claim 4, Frosst in view of Hong disclose the elements of claim 1. In addition, Frosst disclose:
The system of claim 1, wherein there are two or more feature learning tree data structures that are coupled in parallel such that intermediate learned features of a both feature learning tree data structures is concatenated and the concatenation of the intermediate learned features is provided to the decision making tree data structure (In Fig. 2, Frost disclosed two or more feature learning trees that work together at the same time. On page 5 (MINIST Results). The features learned from both trees are combined and then given to the soft decision-making tree.)
Regarding claim 6, Frosst in view of Hong disclose the elements of claim 1. In addition, Frosst disclose:
The system of claim 1, wherein the functions #(x; w), f =f(x; w), and 0(f; w') are linear functions (Fig. 1, Frosst discloses a linear function sigma(wx+b))
Regarding claim 7, Frosst in view of Hong disclose the elements of claim 1. In addition, Frosst disclose:
The system of claim 1, wherein the cascading differentiable tree data structure is a discretized cascading differentiable tree data structure (In Fig. 2, Frosst disclose the process of converting continuous features into discrete ones, using a soft decision tree.)
Regarding claim 8, Frosst in view of Hong disclose the elements of claim 1. In addition, Frosst disclose:
The system of claim 7, wherein only the at least one feature learning tree data structure is discretized (In Fig. 2 and Page 5, Frosst disclose the feature learning tree structure being discretized. )
Regarding claim 10, Frosst in view of Hong disclose the elements of claim 1. In addition, Frosst disclose:
The system of claim 1, wherein each leaf node of the at least one feature learning tree data structure represents one possible assignment of intermediate feature values for a total of L possibilities, and during an inference process, the leaf node with the largest probability is used to assign values for the intermediate features (On page 3, Frosst disclose averaging the distribution over all the leaves, weighted by the respective path probabilities. The predictive distribution from the leaf with the greatest path probability, the explanation for that prediction is simply the list of all the filters along the path from the route to the leaf together with the binary activation decisions.)
With respect to claim 11, Frosst disclose:
A method for automatic conversion of a target machine learning model into differentiable cascading decision tree (CDT)data structure adapted for improved machine learning explainability, the system comprising: a computer processor operating in conjunction with computer memory and a non-transitory computer readable medium, the computer processor configured to (On page 2, The Hierarchical Mixture of Bigots, Frosst discloses a deep neural net into a differentiable decision tree.)
Instantiating the cascading differentiable tree data structure having at least one feature learning tree data structure and a decision making tree data structure, the cascading differentiable tree data structure [[is]] capable of generating an output distribution data structure based on processing a set of data values representing a function of raw features x (On page 2, The Hierarchical Mixture of Bigots, Frosst discloses a differentiable decision tree, where each inner node has a learned filter and a bias, and each leaf node has a learned distribution. Fig. 1 & page 3 disclose a differentiable decision tree that can produce an output distribution based on learnable features f representing a corresponding function sigma (wx+b). Fig. 2 & page 5 further disclose the cascading order of a soft decision tree.)
Iteratively train, until convergence. the cascading differentiable tree data structure against the target machine learning model to mimic representations of the target machine learning model: wherein each of the at least one feature learning tree data structure includes one or more decision nodes that represent the function #(x; w) of raw features x given a set of parameters w and one or more leaf nodes that each represent an intermediate learned feature f based at least on a corresponding intermediate feature representation function f =f(x;w) (In Fig. 1 and page 3, Frosst discloses that each of the at least one feature learning tree data structures includes one or more decision nodes that represent the function sigma(wx+b) of the learnable feature. The soft decision tree is iteratively training the targeted machine learning model to mimic representations using a loss function that seeks to minimize the cross entropy between each leaf, weighted by its path probability, and the target distribution.
Wherein the decision making tree data structure includes one or more decision nodes each representing a corresponding function 0(f; w') of intermediate learned features f given parameters w' from the feature representation functions of the feature learning tree data structure (Fig. 1 & page 3, Frosst disclose a decision tree including one or more nodes each representing a corresponding function sigma(wx+b) of the learnable features given output distribution of Ql or Qr.)
Wherein the at least one feature learning tree data structure is coupled in a cascaded arrangement with the decision making tree data structure (In Fig.2 , Frosst disclose the feature learning tree is connected in a series with the soft decision tree)
With respect to claim 1, Frosst does not explicitly disclose:
Wherein the at least one feature learning tree data structure is coupled in a cascaded arrangement with the decision making tree data structure, and wherein there are two or more feature learning tree data structures that are coupled in the cascaded arrangement such that an intermediate learned feature f of a preceding feature learning tree data structures is provided as a raw input into a subsequent feature learning tree data structure. and the intermediate learned feature f of a final feature learning tree data structure is then provided to the decision making tree data structure
However, it is known by Hong to disclose:
Wherein the at least one feature learning tree data structure is coupled in a cascaded arrangement with the decision making tree data structure, and wherein there are two or more feature learning tree data structures that are coupled in the cascaded arrangement such that an intermediate learned feature f of a preceding feature learning tree data structures is provided as a raw input into a subsequent feature learning tree data structure. and the intermediate learned feature f of a final feature learning tree data structure is then provided to the decision making tree data structure (Fig. 2 & On page 1-2 (Tree Based Feature Representation), Hong disclose Multiple GBDT (Gradient Boosted Decision Tree), outputs/features learned by an earlier tree level are fed into a subsequent tree level. The final cascade output is used for prediction.
Frosst, in view of Hong, they are analogous pieces of art because both references concern Gradient Boosting Decision Trees, which are decision tree ensemble methods exploiting deep cascade structures and using a cross-entropy-based feature representation. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to introduce predictions of a complex deep neural network into a single, easy-to-understand decision tree as taught by Frosst, with multidimensional representational feature vector from preceding level and outputs its processing results to the next level by employing Gradient Boosting Decision Trees (GBDT) models as taught by Hong. The motivation for doing so would have been to improve the interpretability of deep neural networks (CNNs) and the generalization of standard decision trees of Frosst.
Regarding claim 12, Frosst in view of Hong disclose the elements of claim 11. In addition, Frosst disclose:
The method of claim 11, wherein the cascading differentiable tree data structure is adapted to distill the target machine learning architecture by training the cascading differentiable tree data structure based on the target machine learning architecture to mimic an input-output function of the target machine learning architecture (Frosst disclose distilling a neural network into a soft decision tree.)
Regarding claim 13, Frosst in view of Hong disclose the elements of claim 11. In addition, Frosst disclose:
The method of claim 11, wherein depth values of the two or more feature learning tree data structures is a tunable parameter to adjust a balance between the two or more feature learning tree data structure (On page 4, first paragraph, Frosst disclose the size of the tree and then using mini-batch gradient descent to update all of their parameters simultaneously.)
Regarding claim 14, Frosst in view of Hong disclose the elements of claim 11. In addition, Frosst disclose:
The method of claim 11, wherein there are two or more feature learning tree data structures that are coupled in parallel such that intermediate learned features of a both feature learning tree data structures is concatenated and the concatenation of the intermediate learned features is provided to the decision making tree data structure (In Fig. 2, Frosst disclosed two or more feature learning trees that work together at the same time. On page 5 (MINIST Results). The features learned from both trees are combined and then given to the soft decision-making tree.)
Regarding claim 16, Frosst in view of Hong disclose the elements of claim 11. In addition, Frosst disclose:
The method of claim 11, wherein the functions #(x; w), f =f(x; w), and 0(f; w') are linear functions (Fig. 1, Frosst discloses a linear function sigma(wx+b))
Regarding claim 17, Frosst in view of Hong disclose the elements of claim 1. In addition, Frosst disclose:
The system of claim 1, wherein the cascading differentiable tree data structure is a discretized cascading differentiable tree data structure (In Fig. 2, Frosst disclose the process of converting continuous features into discrete ones, using a soft decision tree.)
Regarding claim 18, Frosst disclose the elements of claim 17. In addition, Frosst disclose:
The system of claim 17, wherein only the at least one feature learning tree data structure is discretized (In Fig. 2 and Page 5, Frosst disclose the feature learning tree structure being discretized. )
With respect to claim 20, Frosst disclose:
A non-transitory computer readable medium storing machine interpretable instructions, which when executed by a processor, cause the processor to perform steps of a method for automatic conversion of a target machine learning model into differentiable cascading decision tree (CDT)data structure adapted for improved machine learning explainability, the system comprising: a computer processor operating in conjunction with computer memory and a non-transitory computer readable medium, the computer processor configured to (On page 2, The Hierarchical Mixture of Bigots, Frosst discloses a deep neural net into a differentiable decision tree.)
Instantiating the cascading differentiable tree data structure having at least one feature learning tree data structure and a decision making tree data structure, the cascading differentiable tree data structure [[is]] capable of generating an output distribution data structure based on processing a set of data values representing a function of raw features x (On page 2, The Hierarchical Mixture of Bigots, Frosst discloses a differentiable decision tree, where each inner node has a learned filter and a bias, and each leaf node has a learned distribution. Fig. 1 & page 3 disclose a differentiable decision tree that can produce an output distribution based on learnable features f representing a corresponding function sigma (wx+b). Fig. 2 & page 5 further disclose the cascading order of a soft decision tree.)
Iteratively train, until convergence. the cascading differentiable tree data structure against the target machine learning model to mimic representations of the target machine learning model: wherein each of the at least one feature learning tree data structure includes one or more decision nodes that represent the function #(x; w) of raw features x given a set of parameters w and one or more leaf nodes that each represent an intermediate learned feature f based at least on a corresponding intermediate feature representation function f =f(x;w) (In Fig. 1 and page 3, Frosst discloses that each of the at least one feature learning tree data structures includes one or more decision nodes that represent the function sigma(wx+b) of the learnable feature. The soft decision tree is iteratively training the targeted machine learning model to mimic representations using a loss function that seeks to minimize the cross entropy between each leaf, weighted by its path probability, and the target distribution.
Wherein the decision making tree data structure includes one or more decision nodes each representing a corresponding function 0(f; w') of intermediate learned features f given parameters w' from the feature representation functions of the feature learning tree data structure (Fig. 1 & page 3, Frosst disclose a decision tree including one or more nodes each representing a corresponding function sigma(wx+b) of the learnable features given output distribution of Ql or Qr.)
Wherein the at least one feature learning tree data structure is coupled in a cascaded arrangement with the decision making tree data structure (In Fig.2 , Frosst disclose the feature learning tree is connected in a series with the soft decision tree.)
However, it is known by Hong to disclose:
Wherein the at least one feature learning tree data structure is coupled in a cascaded arrangement with the decision making tree data structure, and wherein there are two or more feature learning tree data structures that are coupled in the cascaded arrangement such that an intermediate learned feature f of a preceding feature learning tree data structures is provided as a raw input into a subsequent feature learning tree data structure. and the intermediate learned feature f of a final feature learning tree data structure is then provided to the decision making tree data structure (Fig. 2 & On page 1-2 (Tree Based Feature Representation), Hong disclose Multiple GBDT (Gradient Boosted Decision Tree), outputs/features learned by an earlier tree level are fed into a subsequent tree level. The final cascade output is used for prediction.
Frosst, in view of Hong, they are analogous pieces of art because both references concern Gradient Boosting Decision Trees, which are decision tree ensemble methods exploiting deep cascade structures and using a cross-entropy-based feature representation. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to introduce predictions of a complex deep neural network into a single, easy-to-understand decision tree as taught by Frosst, with multidimensional representational feature vector from preceding level and outputs its processing results to the next level by employing Gradient Boosting Decision Trees (GBDT) models as taught by Hong. The motivation for doing so would have been to improve the interpretability of deep neural networks (CNNs) and the generalization of standard decision trees of Frosst.
Claims 5, 9, 15 and 19 are rejected under 35 U.S.C. 103 as being unpalatable over Frosst in view of Hong and further in view of Nowozin et al. (Pub No.: 20140122381 A1), hereinafter referred to as Nowozin.
Regarding claim 5, Frosst in view of Hong disclose the elements of claim 1. Frosst in view of Hong do not explicitly disclose:
The system of claim 1, wherein the decision making tree data structure is utilized to process a perturbed set of intermediate feature representation functions to generate a new output distribution data structure, the perturbed set of feature representation functions being perturbed to modify an explainable parameter represented in the perturbed set of intermediate feature representation functions
However, Nowozin disclose the limitation (In paragraph [0059], Nowozin disclose a decision tree training in machine learning, wherein estimates of uncertainty improve the accuracy of predictions and reduce errors when training random decision forests. They use perturbed = measures of randomness, like entropy or Gini index, to produce an improved estimate of the Gini index and with reduced computational complexity.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Frosst and Hong to include Nowozin’s Decision Tree Training to improve uncertainty measures described herein when producing the trained random decision forest as taught by Nowozin (see[0028]).
Regarding claim 9, Frosst in view of Hong disclose the elements of claim 1. Frosst in view of Hong do not explicitly disclose:
The system of claim 1, wherein the cascading differentiable tree data structure is utilized in a random forest of decision tree data structures, and generated outputs of the random forest are utilized in concert to determine an aggregated output distribution data structure
However, Nowozin disclose the limitation (In paragraph [0096-0097], Nowozin disclose training the decision trees, wherein choosing how many decision trees to use in a random decision forest. During training, parameter values (also referred to as features) are learnt for use at the split nodes and data is accumulated at the leaf nodes.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Frosst and Hong to include Nowozin’s Decision Tree Training to improve uncertainty measures described herein when producing the trained random decision forest as taught by Nowozin (see[0028]).
Regarding claim 15, Frosst in view of Hong disclose the elements of claim 11. Frosst in view of Hong do not explicitly disclose:
The method of claim 11, wherein the decision making tree data structure is utilized to process a perturbed set of intermediate feature representation functions to generate a new output distribution data structure, the perturbed set of feature representation functions being perturbed to modify an explainable parameter represented in the perturbed set of intermediate feature representation functions
However, Nowozin disclose the limitation (In paragraph [0059], Nowozin disclose a decision tree training in machine learning, wherein estimates of uncertainty improve the accuracy of predictions and reduce errors when training random decision forests. They use perturbed = measures of randomness, like entropy or Gini index, to produce an improved estimate of the Gini index and with reduced computational complexity.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Frosst and Hong to include Nowozin’s Decision Tree Training to improve uncertainty measures described herein when producing the trained random decision forest as taught by Nowozin (see[0028]).
Regarding claim 19, Frosst in view of Hong disclose the elements of claim 11. Frosst in view of Hong do not explicitly disclose:
The method of claim 11, wherein the cascading differentiable tree data structure is utilized in a random forest of decision tree data structures, and generated outputs of the random forest are utilized in concert to determine an aggregated output distribution data structure
However, Nowozin disclose the limitation (In paragraph [0096-0097], Nowozin disclose training the decision trees, wherein choosing how many decision trees to use in a random decision forest. During training, parameter values (also referred to as features) are learnt for use at the split nodes and data is accumulated at the leaf nodes.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Frosst and Hong to include Nowozin’s Decision Tree Training to improve uncertainty measures described herein when producing the trained random decision forest as taught by Nowozin (see[0028]).
Response to Arguments
The applicant's arguments filed 01/02/2026 have been fully considered, but in part are not persuasive.
Pertaining to Rejection under 101
Applicant’s argument in regard to 101 abstract idea is found persuasive and 101 abstract idea rejection is withdrawn.
Pertaining to Rejection under 103
Applicant’s arguments in regard to the examiner’s rejections under 35 USC 103 are moot in view of the new grounds of rejection.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EVEL HONORE whose telephone number is (703)756-1179. The examiner can normally be reached Monday-Friday 8 a.m. -5:30 p.m.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela D Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
EVEL HONORE
Examiner
Art Unit 2142
/Mariela Reyes/Supervisory Patent Examiner, Art Unit 2142
Prosecution Insights