54 Linear Discriminant Analysis
Logistic regression, covered in the previous topic, models the probability of a class directly and works well for a binary outcome with a modest number of predictors. Linear Discriminant Analysis, LDA (Ronald A. Fisher, 1936), approaches classification from a different angle: it assumes each class’s predictors follow a multivariate normal distribution with its own mean but a shared spread, then classifies a new observation by finding the class whose distribution it most plausibly came from. LDA extends naturally to more than two classes, which logistic regression only does with extra machinery, and tends to work well when the classes really are reasonably normal and well separated in the predictor space.
54.1 How LDA Draws the Boundary
LDA finds a linear combination of the predictors, a discriminant function, that maximizes the separation between class means relative to the spread within each class. Formally it maximizes the ratio of between-class variance to within-class variance:
\[ J(w) = \frac{w^{\mathsf{T}} S_B w}{w^{\mathsf{T}} S_W w} \]
where \(S_B\) is the between-class scatter matrix and \(S_W\) is the within-class scatter matrix. Once fit, a new observation is classified into whichever class’s mean it lies closest to along that discriminant axis, after accounting for each class’s spread. The boundary this produces is a straight line (or a flat plane with more predictors), which is where the “linear” in the name comes from.
54.2 Worked Example
A precision-agriculture setup uses drone imagery to flag crop stress before it is visible to the eye. Twenty fields were scored on NDVI (Normalized Difference Vegetation Index, a standard 0-to-1 measure of vegetation greenness from satellite or drone imagery), soil moisture, and average temperature over the preceding week, then labeled by an agronomist walking the field as Healthy or Stressed.
| Field | NDVI | Soil Moisture (%) | Avg Temp (°C) | Status |
|---|---|---|---|---|
| 1 | 0.42 | 18 | 36 | Stressed |
| 2 | 0.38 | 16 | 37 | Stressed |
| 3 | 0.75 | 38 | 26 | Healthy |
| 4 | 0.68 | 32 | 28 | Healthy |
| 5 | 0.45 | 20 | 34 | Stressed |
| 6 | 0.80 | 40 | 25 | Healthy |
| 7 | 0.55 | 25 | 31 | Stressed |
| 8 | 0.72 | 35 | 27 | Healthy |
| 9 | 0.40 | 17 | 38 | Stressed |
| 10 | 0.78 | 42 | 24 | Healthy |
| 11 | 0.60 | 28 | 30 | Healthy |
| 12 | 0.48 | 22 | 33 | Stressed |
| 13 | 0.82 | 44 | 23 | Healthy |
| 14 | 0.35 | 15 | 38 | Stressed |
| 15 | 0.65 | 30 | 29 | Healthy |
| 16 | 0.50 | 24 | 32 | Stressed |
| 17 | 0.85 | 45 | 22 | Healthy |
| 18 | 0.44 | 19 | 35 | Stressed |
| 19 | 0.58 | 27 | 30 | Healthy |
| 20 | 0.52 | 23 | 33 | Stressed |
This same twenty-field dataset carries through the rest of this topic, so each technique’s result can be compared on identical data.
54.3 LDA in R
MASS ships as part of every standard R installation (it is one of R’s own recommended packages, not an optional add-on), so lda() is available without any extra installation step.
54.4 Reading the Result
The printed model shows the group means for each predictor within Healthy and Stressed fields, and the coefficients of the single discriminant function that separates them (with two classes, LDA needs only one discriminant axis; a third class would add a second). Training accuracy checks how well the fitted boundary recovers the labels it was built from, though as with every technique in this topic, the number that actually matters is accuracy on data the model has not seen, not on its own training set. The new field near the boundary, NDVI around 0.56, is exactly the kind of borderline case where knowing how confidently the model is choosing, not just which class it picked, is worth examining before acting on the prediction.
Summary
| Concept | Description |
|---|---|
| Foundations | |
| Linear Discriminant Analysis | Classifies by assuming each class follows a multivariate normal distribution with a shared spread |
| How LDA Differs from Logistic Regression | LDA models the predictor distribution within each class rather than the outcome probability directly, and extends naturally to more than two classes |
| Between-Class and Within-Class Scatter | LDA maximizes the ratio of between-class separation to within-class spread to find the best boundary |
| The Discriminant Function | A linear combination of predictors that best separates the classes; the boundary itself is a straight line or flat plane |
| Worked Example | |
| Worked Example: NDVI-Based Crop Stress Detection | Twenty fields classified as Healthy or Stressed from NDVI, soil moisture, and temperature |
| Fitting with MASS::lda() | MASS is a recommended package bundled with every standard R installation, so lda() needs no extra setup |
| Classifying a New Observation | predict() applies the fitted discriminant function to classify a field the model has not seen before |