56 Factor Analysis
Factor Analysis (Charles Spearman, 1904) looks similar to PCA on the surface, both take several correlated variables and reduce them to fewer numbers, but the two rest on different assumptions and answer different questions. PCA asks: what combination of these variables captures the most variance, with no claim about why they are correlated? Factor Analysis asks: is there an unobserved, latent variable causing these observed variables to correlate in the first place, and if so, what is it? PCA treats every bit of variance as worth keeping; Factor Analysis explicitly separates the variance each observed variable shares with the underlying factor from the variance that is unique to that one measurement, essentially noise or measurement-specific effects.
\[ X_j = \lambda_j F + \varepsilon_j \]
Each observed variable \(X_j\) is modeled as a loading \(\lambda_j\) on the latent factor \(F\), plus a unique error term \(\varepsilon_j\) that Factor Analysis does not try to explain. This is the same modeling logic used across the social sciences to argue that a single questionnaire measures one underlying construct, applied here to soil chemistry instead of a survey instrument.
56.1 When to Reach for Factor Analysis Instead of PCA
Reach for PCA when the goal is dimensionality reduction for its own sake, building a smaller set of inputs for a later regression or classification step, with no particular claim about an underlying cause. Reach for Factor Analysis when the actual scientific or agronomic question is whether the observed measurements are all surface expressions of one deeper property, in the soil-test example, whether nitrogen, phosphorus, potassium, and organic carbon are all just different windows onto a single latent “soil fertility” factor. The mathematics differ accordingly: PCA has no error term at all, while Factor Analysis explicitly models and separates out each variable’s unique variance.
56.2 Worked Example
The same twelve fields and their soil test results used for PCA.
| Field | N (kg/ha) | P (kg/ha) | K (kg/ha) | Organic Carbon (%) | pH | EC (dS/m) |
|---|---|---|---|---|---|---|
| 1 | 280 | 18 | 140 | 0.65 | 7.2 | 0.35 |
| 2 | 320 | 22 | 160 | 0.75 | 7.0 | 0.30 |
| 3 | 180 | 10 | 90 | 0.40 | 8.1 | 0.55 |
| 4 | 350 | 25 | 175 | 0.82 | 6.8 | 0.28 |
| 5 | 210 | 12 | 100 | 0.45 | 7.8 | 0.48 |
| 6 | 390 | 28 | 190 | 0.90 | 6.6 | 0.25 |
| 7 | 160 | 8 | 75 | 0.35 | 8.3 | 0.60 |
| 8 | 300 | 20 | 150 | 0.70 | 7.1 | 0.32 |
| 9 | 230 | 14 | 110 | 0.50 | 7.6 | 0.45 |
| 10 | 370 | 26 | 182 | 0.85 | 6.7 | 0.27 |
| 11 | 195 | 11 | 95 | 0.42 | 7.9 | 0.52 |
| 12 | 340 | 24 | 168 | 0.78 | 6.9 | 0.29 |
The question Factor Analysis puts to this data: do these six lab measurements really reflect one latent soil-fertility factor, or would they need two or more genuinely separate factors to explain the pattern of correlations among them?
56.3 Factor Analysis in R
factanal(), R’s maximum-likelihood factor analysis function, ships in the base stats package, no additional installation needed.
56.4 Reading the Result
Nitrogen, phosphorus, potassium, and organic carbon should load heavily and in the same direction on the single factor, while pH and EC should load heavily in the opposite direction, echoing the pattern PCA found but now with an explicit causal story attached: one underlying fertility factor is presented as driving all six measurements, and each variable’s uniqueness is the portion of it that this shared factor does not account for. A pH reading is influenced by plenty of soil chemistry beyond generic fertility, so a comparatively large uniqueness for pH would not be surprising and would not by itself argue against the one-factor model. The chi-square test at the end checks the one-factor model’s fit formally: a p-value comfortably above 0.05 supports treating a single latent factor as adequate, while a small p-value would suggest the six measurements need two or more distinct underlying factors to explain their correlation pattern, a genuinely different substantive conclusion than anything PCA alone can offer.
Summary
| Concept | Description |
|---|---|
| Foundations | |
| Factor Analysis | Models observed variables as arising from a smaller number of unobserved latent factors plus unique error |
| The Factor Model Equation | Each observed variable equals a loading on the latent factor plus a variable-specific error term |
| Factor Analysis vs PCA | PCA maximizes variance with no error model; Factor Analysis explicitly separates shared from unique variance |
| When to Use Each | PCA for general dimensionality reduction; Factor Analysis when testing whether one latent construct explains the data |
| Worked Example | |
| Worked Example: Testing a Single Fertility Factor | Testing whether six soil measurements reflect one latent soil-fertility factor across twelve fields |
| Fitting with factanal() | R's base factanal() function fits a maximum-likelihood factor model with no extra package required |
| Factor Loadings and Uniqueness | Loadings show each variable's relationship to the factor; uniqueness shows what the factor leaves unexplained |
| Testing Whether One Factor Is Enough | A chi-square goodness-of-fit test checks whether the chosen number of factors adequately explains the correlations |