62 K-Fold Cross-Validation
Nearly every technique covered in the last two topics came with the same caution attached: evaluate on data the model was not fitted on, not on its own training set. A single train/test split does that, but it has a weakness of its own, the result depends on which observations happened to land in the test set by chance, and with a small agricultural dataset, twenty fields rather than twenty thousand, one unlucky split can make a good model look mediocre or a mediocre model look good. K-fold cross-validation (Mervyn Stone, 1974) fixes this by not relying on any single split at all.
The data is divided into \(k\) roughly equal folds. The model is trained on \(k-1\) of them and evaluated on the fold left out, and this is repeated \(k\) times so that every fold serves as the held-out test set exactly once. The \(k\) resulting performance scores are then averaged, giving a far more stable estimate of how the model is likely to perform on data it has not seen than any single split could.
62.1 Choosing K
\(k = 5\) or \(k = 10\) are the standard choices, trading off two competing concerns. A larger \(k\) means each training fold uses more of the data, which reduces bias in the performance estimate, but the folds overlap more heavily with each other, so the \(k\) resulting scores are more correlated and the variance of the final average is less clearly reduced. At the extreme, \(k\) equal to the sample size is called leave-one-out cross-validation, maximally using the data for training at each step but computationally expensive and, for reasons tied to that correlation between folds, not always the most reliable choice despite using the most data.
62.2 Worked Example
The logistic regression model from Classification Models reported a single training accuracy on the same twenty fields it was fitted on. That number answers “how well does the model fit the data it already saw,” not “how well will it classify a new field,” which is the number that actually matters for deployment. Five-fold cross-validation on the same NDVI, soil moisture, and temperature data gives an honest answer to the second question.
| Field | NDVI | Soil Moisture (%) | Avg Temp (°C) | Status |
|---|---|---|---|---|
| 1 | 0.42 | 18 | 36 | Stressed |
| 2 | 0.38 | 16 | 37 | Stressed |
| 3 | 0.75 | 38 | 26 | Healthy |
| 4 | 0.68 | 32 | 28 | Healthy |
| 5 | 0.45 | 20 | 34 | Stressed |
| 6 | 0.80 | 40 | 25 | Healthy |
| 7 | 0.55 | 25 | 31 | Stressed |
| 8 | 0.72 | 35 | 27 | Healthy |
| 9 | 0.40 | 17 | 38 | Stressed |
| 10 | 0.78 | 42 | 24 | Healthy |
| 11 | 0.60 | 28 | 30 | Healthy |
| 12 | 0.48 | 22 | 33 | Stressed |
| 13 | 0.82 | 44 | 23 | Healthy |
| 14 | 0.35 | 15 | 38 | Stressed |
| 15 | 0.65 | 30 | 29 | Healthy |
| 16 | 0.50 | 24 | 32 | Stressed |
| 17 | 0.85 | 45 | 22 | Healthy |
| 18 | 0.44 | 19 | 35 | Stressed |
| 19 | 0.58 | 27 | 30 | Healthy |
| 20 | 0.52 | 23 | 33 | Stressed |
62.3 K-Fold Cross-Validation in R
62.4 Reading the Result
The naive training accuracy and the mean cross-validated accuracy will rarely match exactly, and the gap between them is itself the headline finding: it is a direct estimate of how optimistic the training-set number was. With only twenty fields split five ways, each fold holds just four observations, so expect real variability in the per-fold accuracy figures; the standard deviation across folds is reporting exactly that instability, not a flaw in the method. On a genuinely small agricultural dataset like this one, that variability is honest information a single train/test split would have hidden entirely, and it is exactly the kind of uncertainty a farm advisory tool should account for before treating any one field’s prediction as certain.
Summary
| Concept | Description |
|---|---|
| Foundations | |
| K-Fold Cross-Validation | Rotates through k folds so every observation serves as held-out test data exactly once, then averages the results |
| Why a Single Train/Test Split Falls Short | A single split's performance estimate depends heavily on which observations happened to land in the test set |
| Choosing K | k = 5 or k = 10 are standard, balancing how much data each fold trains on against how correlated the folds become |
| Leave-One-Out Cross-Validation | Uses one observation per fold at the extreme; maximizes training data but is expensive and not always most reliable |
| Worked Example | |
| Worked Example: Revisiting the Logistic Regression Classifier | Cross-validating the NDVI-based crop health classifier from the Classification Models topic |
| Implementing K-Fold CV in R | A manual fold loop refits the model k times and averages accuracy across the held-out folds |
| Naive vs Cross-Validated Accuracy | The gap between training-set accuracy and cross-validated accuracy estimates how optimistic the training number was |
| Variability Across Folds | Wide variation across folds is itself informative, especially with a small dataset |