69  AR, MA, ARMA and ARIMA Forecasting

Hidden Markov Models, covered in the previous section, inferred an unobserved state from a sequence of observations. This section stays with sequential, time-ordered data but asks a different question: given a variable’s own past values, a commodity’s monthly mandi price, say, what will it do next? The AR, MA, ARMA, and ARIMA family of models (George E. P. Box & Gwilym M. Jenkins, 1970) forecasts a series purely from its own history, with no other predictor variables required at all.

These are not four unrelated techniques requiring four separate treatments. They are nested special cases of a single model, ARIMA(p, d, q), distinguished by which of its three components is switched on:

  • AR(p), autoregression: the current value is a weighted combination of the previous \(p\) values of the series itself. This is ARIMA(p, 0, 0).
  • MA(q), moving average: the current value is a weighted combination of the previous \(q\) forecast errors, not the previous values. This is ARIMA(0, 0, q).
  • ARMA(p, q): combines both an autoregressive and a moving-average component. This is ARIMA(p, 0, q).
  • ARIMA(p, d, q): adds a third component, \(d\), the number of times the series is differenced (each value replaced by its change from the previous value) before AR and MA terms are fit, needed whenever the raw series is not stationary.

69.1 Stationarity and Differencing

AR and MA models assume the series is stationary: its mean, variance, and autocorrelation structure stay constant over time, rather than drifting in a trend or shifting to a new level. Most real commodity price series are not stationary on their own; prices trend upward with inflation and input costs over several years, which is exactly the kind of drift that violates the assumption. Differencing, replacing each observation with the difference from the one before it, often removes a trend and leaves a series stationary enough to model. \(d = 1\) (one round of differencing) handles most linear trends; a series that still trends after one round of differencing may need \(d = 2\), though this is uncommon in practice.

Two diagnostic plots guide the choice of \(p\) and \(q\) once the series is stationary: the autocorrelation function (ACF), which shows how strongly the series correlates with lagged versions of itself, and the partial autocorrelation function (PACF), which shows that same correlation after removing the effect of the shorter lags in between. A PACF that cuts off sharply after lag \(p\) suggests an AR(p) term; an ACF that cuts off sharply after lag \(q\) suggests an MA(q) term. In practice, several candidate orders are usually fit and compared by a criterion such as AIC (Akaike Information Criterion) rather than read off the plots alone.

69.2 Worked Example

Four years (48 months) of wholesale tomato prices at a district mandi, showing a mild upward trend, a repeating seasonal pattern tied to the harvest calendar, and month-to-month noise, the typical shape of an agricultural commodity price series. The last 6 months are held out to check forecast accuracy against actual, already-known prices.

69.3 ARIMA Forecasting in R

69.4 Reading the Result

The first-half-versus-second-half mean comparison should show a visible gap, confirming the trend that motivated differencing in the first place, and the differenced series’ mean should sit much closer to zero, evidence that one round of differencing did its job. The AIC comparison is doing the same job the elbow method did for choosing \(k\) in the previous topic: rather than committing to a single order on theoretical grounds alone, several reasonable candidates are fit and compared, and the one balancing fit quality against model complexity wins. Note that whichever order the AIC favors, the model is still only ARIMA with specific \((p, d, q)\) values, the same underlying framework as the AR and MA special cases described earlier, just with both components switched on together.

The forecast RMSE puts a number on how far the six-month-ahead forecast landed from what actually happened, the same honest, held-out evaluation principle used throughout the model validation topic earlier in this unit. A forecast that tracks the seasonal turn reasonably well but drifts further from actual prices in later months is typical of this family of models: uncertainty compounds the further out a forecast reaches, which is exactly why commodity price forecasts quoted with confidence several seasons out deserve more scrutiny than one quoted a single month ahead.

Summary

Concept Description
Foundations
The AR/MA/ARMA/ARIMA Family Nested special cases of a single model, ARIMA(p,d,q), forecasting a series from its own past values alone
Autoregression (AR) AR(p): the current value depends on a weighted combination of the previous p values of the series
Moving Average (MA) MA(q): the current value depends on a weighted combination of the previous q forecast errors
ARIMA as the General Case ARIMA(p,d,q) adds d rounds of differencing to AR and MA components, handling non-stationary series
Worked Example
Stationarity and Differencing AR and MA require a stationary series; differencing removes a trend so the assumption holds reasonably well
ACF and PACF for Order Selection The autocorrelation and partial autocorrelation functions guide, but do not fully determine, the choice of p and q
Worked Example: Forecasting a Mandi Price Series Four years of monthly tomato mandi prices with trend, seasonality, and noise, forecast six months ahead
Evaluating a Forecast on Held-Out Months Comparing forecasts against six held-out actual months via RMSE, the same honest evaluation principle as before