60 Random Forest
A single decision tree is unstable: change the training data slightly, a few different fields in the sample, and the tree that comes out can look quite different, since an early split near the top reshapes everything beneath it. Random Forest (Leo Breiman, 2001) fixes this by growing many trees instead of one, each on a bootstrap-resampled version of the data (a random sample of the same size, drawn with replacement, so some fields appear more than once and others not at all), and each considering only a random subset of the predictors at every split rather than all of them. Every tree in the forest ends up a little different, and the forest’s final prediction is a majority vote across all of them.
This combination, bootstrap resampling of observations plus random restriction of predictors, is what separates a random forest from plain bagging of identical trees: it deliberately decorrelates the individual trees from each other, so their errors are less likely to all point the same wrong direction at once.
60.1 Why Averaging Many Trees Helps
A single deep tree tends to have low bias (it can fit the training data very closely) but high variance (it changes a lot with small changes to the data), the overfitting risk flagged at the end of the previous section. Averaging predictions across many trees, provided they are not all making the same mistakes, keeps the low bias of individual trees while substantially cutting the variance. This only works because the trees are genuinely different from each other, which is exactly why random forest bothers with both bootstrap resampling and random predictor subsets rather than just growing the same tree on the same data repeatedly.
60.2 Worked Example
The same twenty fields used throughout this topic.
| Field | NDVI | Soil Moisture (%) | Avg Temp (°C) | Status |
|---|---|---|---|---|
| 1 | 0.42 | 18 | 36 | Stressed |
| 2 | 0.38 | 16 | 37 | Stressed |
| 3 | 0.75 | 38 | 26 | Healthy |
| 4 | 0.68 | 32 | 28 | Healthy |
| 5 | 0.45 | 20 | 34 | Stressed |
| 6 | 0.80 | 40 | 25 | Healthy |
| 7 | 0.55 | 25 | 31 | Stressed |
| 8 | 0.72 | 35 | 27 | Healthy |
| 9 | 0.40 | 17 | 38 | Stressed |
| 10 | 0.78 | 42 | 24 | Healthy |
| 11 | 0.60 | 28 | 30 | Healthy |
| 12 | 0.48 | 22 | 33 | Stressed |
| 13 | 0.82 | 44 | 23 | Healthy |
| 14 | 0.35 | 15 | 38 | Stressed |
| 15 | 0.65 | 30 | 29 | Healthy |
| 16 | 0.50 | 24 | 32 | Stressed |
| 17 | 0.85 | 45 | 22 | Healthy |
| 18 | 0.44 | 19 | 35 | Stressed |
| 19 | 0.58 | 27 | 30 | Healthy |
| 20 | 0.52 | 23 | 33 | Stressed |
60.3 Random Forest in R
The dedicated randomForest package is not part of R’s base or recommended set and may not be available in every environment. The forest below is built directly from its own definition instead: bootstrap resampling plus random predictor subsets, using rpart (recommended, always available) as the individual tree learner.
60.4 Reading the Result
The vote count for each field is worth more attention than the majority label alone: a field with 14 out of 15 trees agreeing is a confident call, while one splitting 8 to 7 is genuinely borderline, information a single tree’s flat classification cannot offer at all. This vote share is the random forest analogue of the predicted probability logistic regression and Naive Bayes report directly, and it is exactly what should inform where a threshold gets set if flagging a field for a costly field visit is the downstream action, not just whether the raw majority tips one way or the other.
Summary
| Concept | Description |
|---|---|
| Foundations | |
| Random Forest | Grows many decision trees on resampled data and predictor subsets, then combines them by majority vote |
| Bootstrap Resampling | Each tree is trained on a random sample of the same size drawn with replacement from the original data |
| Random Predictor Subsets | Each split considers only a random subset of predictors, decorrelating the trees from each other |
| Why Averaging Many Trees Reduces Variance | Low-bias, high-variance individual trees average into a lower-variance prediction, provided they genuinely differ |
| Worked Example | |
| Worked Example: Crop Health Classification | The same twenty NDVI, soil moisture, and temperature fields classified as Healthy or Stressed |
| Building a Forest from rpart Trees | A forest of fifteen rpart trees built by hand from bootstrap samples and random two-of-three predictor subsets |
| Majority Voting | The class predicted by the largest number of trees becomes the forest's final prediction for that observation |
| Vote Share as a Confidence Signal | The proportion of trees agreeing on a prediction indicates how confident or borderline that call actually is |