Bagging
Bootstrap N samples with replacement per tree. Reduces variance. ~63.2% unique samples per tree.
Advertisement
Random subspace
At each split, consider only √d (classification) or d/3 (regression) random features. Decorrelates trees.
Advertisement
Predictions
Classification: majority vote. Regression: average. Out-of-bag samples give free validation.
Feature importance
Gini importance: total impurity decrease per feature. Permutation importance: shuffle feature, measure accuracy drop.