Random Forests Deep Dive
Ensemble of decision trees.
Bootstrap Sampling
Sample with replacement. Build each tree on different data.
Feature Randomness
Random subset of features at each split. Reduces correlation between trees.
Aggregation
Majority vote for classification. Average for regression. Reduces overfitting.
Key Takeaways
- Ensemble of decorrelated trees
- Bootstrap + feature randomness
- More robust than single tree