55 Principal Component Analysis
A standard soil test returns half a dozen correlated numbers at once: nitrogen, phosphorus, potassium, organic carbon, pH, electrical conductivity. Fertile soils tend to be high in the first four and lower in the last two, so these six numbers do not carry six independent pieces of information, they carry a smaller number of underlying patterns measured six overlapping ways. Principal Component Analysis, PCA (Karl Pearson, 1901), finds those underlying patterns directly: it re-expresses a set of correlated variables as a smaller set of uncorrelated ones, called principal components, each a weighted combination of the originals, ordered so the first component captures as much of the total variation as possible, the second captures as much of what remains, and so on.
Unlike every technique elsewhere in this topic, PCA is unsupervised: there is no outcome variable being predicted, only structure being found in the predictors themselves.
55.1 What a Principal Component Is
The first principal component is the direction through the data that maximizes variance:
\[ PC_1 = w_1 X_1 + w_2 X_2 + \cdots + w_p X_p, \qquad \text{subject to} \sum_{j=1}^{p} w_j^2 = 1 \]
The weights \(w_j\) are chosen so that projecting every observation onto this single direction preserves as much of the original spread as possible. Each subsequent component repeats this, constrained to be uncorrelated with every component already found. Variables need to be standardized before running PCA, since a variable measured in the hundreds (nitrogen in kg/ha) would otherwise dominate the variance calculation purely because of its scale, not because it actually varies more meaningfully than pH.
55.2 Worked Example
Twelve fields were soil-tested for six standard fertility indicators.
| Field | N (kg/ha) | P (kg/ha) | K (kg/ha) | Organic Carbon (%) | pH | EC (dS/m) |
|---|---|---|---|---|---|---|
| 1 | 280 | 18 | 140 | 0.65 | 7.2 | 0.35 |
| 2 | 320 | 22 | 160 | 0.75 | 7.0 | 0.30 |
| 3 | 180 | 10 | 90 | 0.40 | 8.1 | 0.55 |
| 4 | 350 | 25 | 175 | 0.82 | 6.8 | 0.28 |
| 5 | 210 | 12 | 100 | 0.45 | 7.8 | 0.48 |
| 6 | 390 | 28 | 190 | 0.90 | 6.6 | 0.25 |
| 7 | 160 | 8 | 75 | 0.35 | 8.3 | 0.60 |
| 8 | 300 | 20 | 150 | 0.70 | 7.1 | 0.32 |
| 9 | 230 | 14 | 110 | 0.50 | 7.6 | 0.45 |
| 10 | 370 | 26 | 182 | 0.85 | 6.7 | 0.27 |
| 11 | 195 | 11 | 95 | 0.42 | 7.9 | 0.52 |
| 12 | 340 | 24 | 168 | 0.78 | 6.9 | 0.29 |
Nitrogen, phosphorus, potassium, and organic carbon rise and fall together across the twelve fields, while pH and electrical conductivity move the opposite way, exactly the kind of overlapping structure PCA is meant to compress.
55.3 Principal Component Analysis in R
55.4 Reading the Result
The first component should capture a large share of the total variance on its own, typically 70 to 90 percent for a dataset built around a single dominant fertility pattern like this one. Its loadings should show nitrogen, phosphorus, potassium, and organic carbon with the same sign and pH and EC with the opposite sign, meaning a field’s score on this one component summarizes its overall fertility about as well as all six original measurements combined. That single score, often labeled a soil fertility index in agronomic practice, is what PCA was built to produce: a smaller number of variables that still carries most of the original information, useful both for simpler reporting to a farmer and as a cleaned-up input to a downstream regression or classification model where six correlated soil variables would otherwise reintroduce the multicollinearity problem covered in the previous topic.
Summary
| Concept | Description |
|---|---|
| Foundations | |
| Principal Component Analysis | Re-expresses correlated variables as a smaller set of uncorrelated components ordered by variance explained |
| PCA Is Unsupervised | PCA finds structure in the predictors alone, with no outcome variable being predicted |
| The First Principal Component | The direction through standardized data that captures the maximum possible variance |
| Why Standardization Is Required | Variables on different scales would otherwise distort the variance calculation before it even starts |
| Worked Example | |
| Worked Example: Soil Fertility Indicators | Six correlated soil fertility measurements across twelve fields reduced to their underlying pattern |
| Fitting with prcomp() | R's base prcomp() function with scale. = TRUE performs PCA on standardized variables |
| Proportion of Variance Explained | Shows how much of the original information each successive component preserves |
| PCA as a Fertility Index | A field's score on the first component summarizes overall fertility from all six original measurements |