39  Analysis of Variance (ANOVA) and the F-Test

ANOVA (Analysis of Variance) is a statistical technique used to determine whether there are significant differences among the means of three or more independent groups. Rather than comparing means directly, ANOVA compares the variance between groups to the variance within groups, and the ratio it produces is called the F-statistic, the test statistic of the F-test. So “the F-test” and “ANOVA” are two names pointing at the same procedure: ANOVA is the framework, and the F-test is the significance test it runs. The goal is to detect whether any group differs, not which one. That’s what post-hoc tests are for.

39.0.1 Types of ANOVA

  1. One-Way ANOVA: tests the effect of a single factor on a continuous outcome across two or more groups, e.g., the effect of three seed treatment methods on yield.
  2. Two-Way ANOVA: tests the effect of two factors simultaneously, including whether they interact, e.g., the joint effect of fertilizer type and irrigation method on plant height.
  3. Repeated Measures ANOVA: used when the same subjects are measured under every condition, e.g., the same field’s soil moisture tracked across four growth stages.
  4. MANOVA (Multivariate ANOVA): extends ANOVA to two or more dependent variables analyzed together.

39.0.2 Assumptions

  • Independence of cases: groups consist of different, unrelated observations.
  • Normality: residuals (observed minus predicted values) should be approximately normally distributed.
  • Homogeneity of variances: variance should be roughly equal across groups. Checked with Levene’s or Bartlett’s test.

39.0.3 The F-Statistic and Steps to Conduct ANOVA

\[F = \frac{\text{Variance between groups}}{\text{Variance within groups}} = \frac{MSB}{MSW}\]

  1. State the hypotheses: \(H_0\) (all group means are equal; \(H_a\)) at least one group mean differs.
  2. Compute the F-statistic from the between-group and within-group variances.
  3. Compare to the F-distribution at the chosen significance level, using degrees of freedom for the numerator (between) and denominator (within).
  4. Decide: if \(F\) exceeds the critical value (equivalently, if \(p < \alpha\)), reject \(H_0\).

39.0.4 Post-Hoc Tests and Effect Size

A significant ANOVA only says some group differs. It doesn’t say which. Post-hoc tests such as Tukey’s HSD, Bonferroni, or Dunnett’s identify the specific pairs that differ, correcting for the extra false-positive risk of making multiple comparisons.

As with the t-test, statistical significance isn’t the whole story. Eta-squared (\(\eta^2\)) reports how much of the total variance the factor actually explains:

\[\eta^2 = \frac{SSB}{SST}\]

Rough conventions: \(\eta^2 \approx 0.01\) small, \(0.06\) medium, \(0.14\) or above large.

39.1 One-Way ANOVA

One-way ANOVA compares the means of three or more independent groups on a single factor. It extends the t-test to more than two groups without inflating the Type I error rate that repeated pairwise t-tests would cause.

Hypotheses

  • \(H_0\): all group means are equal.
  • \(H_1\): at least one group mean differs from the others.

Example: Yield Across Three Seed Treatment Methods

An agricultural cooperative wants to know whether three seed treatment methods (untreated (control), fungicide-coated, and bio-primed) affect yield (quintals/acre). Fifteen plots were randomly assigned, five to each method:

Plot Untreated Fungicide-Coated Bio-Primed
1 11 17 15
2 15 18 16
3 18 21 18
4 19 22 19
5 22 27 22

At \(\alpha = 0.05\), do the three treatment methods differ in effect on yield?

Calculation

Group means: \(\bar{x}_{Untreated} = 17\), \(\bar{x}_{Fungicide} = 21\), \(\bar{x}_{Bio\text{-}primed} = 18\). Grand mean: \(\bar{x} = 18.667\).

\[SSB = 5\left[(17-18.667)^2 + (21-18.667)^2 + (18-18.667)^2\right] = 5(8.667) = 43.33\]

\[SSW = \sum(x_{i}-\bar{x}_{group})^2 \text{ across all 15 plots} = 162.0, \qquad SST = SSB + SSW = 205.33\]

\[MSB = \frac{43.33}{3-1} = 21.67, \qquad MSW = \frac{162.0}{15-3} = 13.5, \qquad F = \frac{21.67}{13.5} = 1.61\]

\(df_1 = 2\), \(df_2 = 12\). The critical F-value at \(\alpha=0.05\) is about 3.89. Since \(1.61 < 3.89\), we fail to reject \(H_0\) (\(p \approx 0.24\)).

Effect size: \(\eta^2 = \dfrac{43.33}{205.33} \approx 0.21\), a large proportion of variance explained by treatment method, even though the test itself isn’t significant with only 5 plots per group. That combination is a real and common finding: a promising-looking effect that a small sample can’t yet confirm statistically. It argues for a larger trial, not for dismissing the pattern.

One-Way ANOVA in R

39.2 Two-Way ANOVA

Two-way ANOVA extends one-way ANOVA to two factors at once, allowing a test not only of each factor’s own effect but of whether the two factors interact, whether one factor’s effect depends on the level of the other.

Hypotheses

A two-way ANOVA tests three null hypotheses simultaneously:

  1. Main effect of Factor A: no difference in outcome across A’s levels.
  2. Main effect of Factor B: no difference in outcome across B’s levels.
  3. Interaction (A × B): the effect of A on the outcome doesn’t depend on B’s level, and vice versa.

Example: Fertilizer Type and Irrigation Method on Plant Height

A study evaluates the effect of Fertilizer Type (A, B) and Irrigation Method (X, Y) on plant height (cm):

  • Fertilizer A, Irrigation X: 15, 17, 16
  • Fertilizer A, Irrigation Y: 14, 15, 15
  • Fertilizer B, Irrigation X: 18, 20, 19
  • Fertilizer B, Irrigation Y: 22, 21, 23

Results

Source Sum of Squares F p-value \(\eta^2\)
Fertilizer Type 80.08 96.1 < 0.0001 0.778
Irrigation Method 2.08 2.5 0.153 0.020
Interaction 14.08 16.9 0.003 0.137
  • Fertilizer type has a highly significant, large effect on plant height (\(\eta^2 = 0.78\)). Reject \(H_{0a}\).
  • Irrigation method alone shows no significant effect (\(p = 0.153\)). Fail to reject \(H_{0b}\).
  • The interaction is significant (\(p = 0.003\), \(\eta^2 = 0.14\)): irrigation method’s effect on height depends on which fertilizer is used, and vice versa. Because the interaction is significant, the main effects shouldn’t be read in isolation. The best combination of factors depends on both together, not either alone.

Two-Way ANOVA in R

aov() is part of base R’s stats package. No extra packages are needed for a two-way ANOVA.