Decision Trees¶

Table of Contents

Classification and Regression Trees (CART)
Bagging
- Bootstrap Samples
Random Forest
- Variance Reduction in Averaging Correlated Models
- Algorithm Outline
Boosting
Resources

Classification and Regression Trees (CART)¶

Motivation: Approximating Bayes Estimators¶

Inhomogeneity (Impurity) Measure¶

Categorical Predictors¶

Bagging¶

Bootstrap Samples¶

Random Forest¶

Variance Reduction in Averaging Correlated Models¶

Algorithm Outline¶

Practical Suggestions¶

Relative Feature Importance¶

Proximity Plot¶

Boosting¶

AdaBoost.M1: Algorithm Outline¶

Forward Stage-wise Additive Modeling¶

AdaBoost as a Additive Model¶

Exponential Loss¶

Robust Loss Functions¶

Gradient Boosting: Algorithm Outline¶

Relative Feature Importance¶

Resources¶

Note

Trees: https://scikit-learn.org/stable/modules/tree.html
Ensemble: https://scikit-learn.org/stable/modules/ensemble.html