43 Linear Regression
Linear regression fits a straight line through a set of paired observations to predict a continuous outcome from a single predictor. It is the simplest regression model and the one every other technique in this topic builds on or departs from in some specific way. The fitted line takes the form:
\[ \hat{Y} = b_0 + b_1 X \]
Here \(b_0\) is the intercept (the predicted value of \(Y\) when \(X\) is zero) and \(b_1\) is the slope (how much \(Y\) changes for each one-unit increase in \(X\)). Both are chosen by the method of least squares, which picks the line that minimizes the sum of squared vertical distances between the observed points and the line itself.
43.1 The Least-Squares Calculation
\[ b_1 = \frac{\sum (X_i - \bar{X})(Y_i - \bar{Y})}{\sum (X_i - \bar{X})^2} \qquad\qquad b_0 = \bar{Y} - b_1 \bar{X} \]
\(b_1\) is the sum of cross-products of deviations from the two means, divided by the sum of squared deviations of \(X\) from its mean. Once \(b_1\) is known, \(b_0\) follows directly: the fitted line always passes through the point \((\bar{X}, \bar{Y})\).
43.2 Worked Example
An agronomist wants to know how nitrogen fertilizer dose affects wheat yield on a research plot. Eight sub-plots received different nitrogen doses, and yield was measured at harvest.
| Sub-plot | Nitrogen Dose (kg/ha) | Yield (quintals/ha) |
|---|---|---|
| 1 | 40 | 24 |
| 2 | 60 | 27 |
| 3 | 80 | 30 |
| 4 | 100 | 33 |
| 5 | 120 | 35 |
| 6 | 140 | 37 |
| 7 | 160 | 39 |
| 8 | 180 | 39 |
Calculate the Means
\[ \bar{X} = \frac{40+60+80+100+120+140+160+180}{8} = 110 \qquad \bar{Y} = \frac{24+27+30+33+35+37+39+39}{8} = 33 \]
Calculate the Slope
| \(X_i - \bar{X}\) | \(Y_i - \bar{Y}\) | Product | \((X_i-\bar{X})^2\) |
|---|---|---|---|
| -70 | -9 | 630 | 4900 |
| -50 | -6 | 300 | 2500 |
| -30 | -3 | 90 | 900 |
| -10 | 0 | 0 | 100 |
| 10 | 2 | 20 | 100 |
| 30 | 4 | 120 | 900 |
| 50 | 6 | 300 | 2500 |
| 70 | 6 | 420 | 4900 |
\[ \sum (X_i-\bar{X})(Y_i-\bar{Y}) = 1880 \qquad \sum (X_i-\bar{X})^2 = 16800 \]
\[ b_1 = \frac{1880}{16800} \approx 0.112 \]
Calculate the Intercept and Final Equation
\[ b_0 = 33 - (0.112 \times 110) \approx 33 - 12.32 = 20.68 \]
The fitted regression equation is:
\[ \hat{Y} = 20.68 + 0.112\,X \]
Interpretation: each additional kilogram of nitrogen per hectare is associated with about 0.112 quintals more wheat yield per hectare. A dose of 150 kg/ha would be predicted to yield \(20.68 + 0.112 \times 150 \approx 37.5\) quintals per hectare, close to what sub-plots 6 and 7 actually produced, a reasonable sign that the line fits the middle of the data well even though no sub-plot was tested at exactly 150 kg/ha.
43.3 Linear Regression in R
43.4 Where a Straight Line Falls Short
A straight line worked reasonably well for this dataset because yield kept climbing roughly in step with nitrogen across the range tested. That will not hold if the dose range is pushed wider: past a certain point, nitrogen stops helping and can even hurt yield through lodging or nutrient imbalance. The next three sections in this topic (multiple, polynomial, and nonlinear regression) exist precisely because a single straight line cannot capture more than one predictor or a curve that bends. Quantile regression, covered last, addresses a different limitation altogether: it predicts a chosen percentile of yield rather than the average, which matters when a farmer cares less about the typical outcome than about what a poor season is likely to still deliver.
Summary
| Concept | Description |
|---|---|
| Foundations | |
| Linear Regression | Fits a straight line predicting a continuous outcome from a single predictor |
| Least-Squares Slope and Intercept | b1 is the sum of cross-product deviations divided by the sum of squared X deviations; b0 follows from the two means |
| Worked Example | |
| Worked Example: Nitrogen and Yield | Fitting nitrogen dose against wheat yield across 8 sub-plots by hand and in R |
| Interpreting the Slope | Each additional unit of X is associated with a b1 change in predicted Y |
| Prediction from the Fitted Line | Substituting a new X value into the fitted equation to forecast Y |
| Moving Forward | |
| Limits of a Straight Line | A straight line cannot capture multiple predictors, curvature, or outcomes beyond the average |