52 Logistic Regression
Every regression technique covered so far in this unit predicts a continuous number: a yield in quintals per hectare, a residue level in parts per million. Many decisions a farm advisor or an agribusiness needs to make are not continuous at all. Will this field develop a pest outbreak this season, yes or no? Will a loan applicant default, yes or no? Will a batch of produce pass a quality inspection, yes or no? These are classification problems, and fitting an ordinary regression line to a 0/1 outcome breaks down immediately: the fitted line can predict values below 0 or above 1, which have no meaning as a probability, and the outcome’s variance is not constant the way ordinary least squares assumes.
Logistic regression (David R. Cox, 1958) solves this by modeling the log-odds of the outcome as a linear function of the predictors, rather than modeling the outcome directly:
\[ \ln\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_k X_k \]
where \(p\) is the probability that the outcome equals 1. Solving this equation for \(p\) gives the logistic (sigmoid) function:
\[ p = \frac{1}{1 + e^{-(\beta_0 + \beta_1 X_1 + \cdots + \beta_k X_k)}} \]
This function is bounded between 0 and 1 for any input, which is exactly the property a probability needs and a straight line does not have.
52.1 Odds, Log-Odds, and Coefficient Interpretation
The odds of an event are the probability it happens divided by the probability it does not: \(\text{odds} = p / (1-p)\). A probability of 0.75 corresponds to odds of 3 to 1. Logistic regression coefficients are most naturally read on the odds ratio scale, obtained by exponentiating a coefficient: \(e^{\beta_j}\) gives the multiplicative change in the odds of the outcome for a one-unit increase in \(X_j\), holding the other predictors fixed. An odds ratio above 1 means the predictor raises the odds of the outcome; below 1 means it lowers them; exactly 1 means no effect.
Coefficients themselves are estimated by maximum likelihood rather than least squares, since there is no closed-form formula analogous to the normal equations used for linear regression. In practice this search is handled by the fitting software (R’s glm() function below) through an iterative algorithm, not calculated by hand.
52.2 Worked Example
An extension officer wants to flag which fields are at elevated risk of a pest outbreak partway through the season, using two readily available signals: average humidity over the preceding two weeks and the number of days since the field was last sprayed. Sixteen fields were monitored, and each was recorded as having an outbreak (1) or not (0) by the end of the season.
| Field | Humidity (%) | Days Since Last Spray | Pest Outbreak |
|---|---|---|---|
| 1 | 55 | 5 | No |
| 2 | 60 | 8 | No |
| 3 | 62 | 20 | No |
| 4 | 65 | 10 | No |
| 5 | 68 | 25 | No |
| 6 | 70 | 15 | No |
| 7 | 72 | 30 | Yes |
| 8 | 75 | 18 | No |
| 9 | 78 | 35 | Yes |
| 10 | 80 | 22 | Yes |
| 11 | 82 | 40 | Yes |
| 12 | 85 | 28 | Yes |
| 13 | 88 | 45 | Yes |
| 14 | 90 | 32 | Yes |
| 15 | 92 | 50 | Yes |
| 16 | 95 | 38 | Yes |
Outbreaks cluster at higher humidity and more days since spraying, but the boundary is not perfectly clean, field 6 stayed outbreak-free despite moderate humidity, and field 7 had one despite a shorter gap since spraying than several outbreak-free fields. That overlap is realistic, and it is exactly the kind of pattern logistic regression is built to summarize as a probability rather than a hard rule.
52.3 Logistic Regression in R
52.4 Reading the Result
Both coefficients should come out positive, meaning higher humidity and a longer gap since the last spray each raise the log-odds of an outbreak, matching what the raw table already suggested. Converting to odds ratios makes the size of each effect easier to communicate to a non-statistician: an odds ratio of, say, 1.15 for humidity means each additional percentage point of humidity multiplies the odds of an outbreak by about 1.15, holding the spray gap fixed. The predicted probability for the new field falls somewhere between 0 and 1 by construction, and the extension officer still has to decide what probability counts as “risky enough to act on,” which is exactly the threshold question the next section takes up.
Summary
| Concept | Description |
|---|---|
| Foundations | |
| Why Linear Regression Fails for a 0/1 Outcome | A straight line can predict values outside 0 to 1, which have no meaning as a probability |
| The Logit and Sigmoid Functions | The logit models log-odds as linear in the predictors; the sigmoid inverts this back to a bounded probability |
| Odds and Log-Odds | Odds equal probability divided by one minus probability; log-odds is the natural log of the odds |
| Odds Ratio Interpretation | Exponentiating a coefficient gives the multiplicative change in odds for a one-unit increase in that predictor |
| Maximum Likelihood Estimation | Logistic regression coefficients are fit by an iterative likelihood search, not a closed-form formula |
| Worked Example | |
| Worked Example: Pest Outbreak Risk | Humidity and days since last spray predicting pest outbreak across sixteen monitored fields |
| Fitting with glm() in R | R's glm() with family = binomial fits a logistic regression model directly |
| Predicting a Probability for a New Case | predict() with type = 'response' returns a probability between 0 and 1 for a new observation |