ARIMA (AutoRegressive Integrated Moving Average)

ARIMA is a family of time-series models built from three ideas. The name is unpacked in the standard reference: “if we combine differencing with autoregression and a moving average model, we obtain a non-seasonal ARIMA model. ARIMA is an acronym for AutoRegressive Integrated Moving Average (in this context, ‘integration’ is the reverse of differencing).”

A model is written ARIMA(\(p,d,q\)) where \(p\) is “order of the autoregressive part,” \(d\) the “degree of first differencing involved,” and \(q\) the “order of the moving average part.” Each letter answers a different question about what the model remembers.

Component	Uses	Purpose
AR(\(p\))	Past values of the series	Persistence — today resembles recent days
I(\(d\))	Differences between consecutive values	Removing level and trend to reach stationarity
MA(\(q\))	Past errors	Shocks whose effect persists for a few periods

The “predictors” in the model “include both lagged values of \(y_t\) and lagged errors,” which is what distinguishes it from an ordinary regression on time.

Stationarity is the precondition

“A stationary time series is one whose statistical properties do not depend on the time at which the series is observed,” so “time series with trends, or with seasonality, are not stationary.” One case is worth knowing because it catches people: a series with cyclic behavior but no trend or seasonality can still be stationary, “because the cycles are not of a fixed length,” so the timing of the peaks and troughs cannot be predicted from the calendar. What has to hold is that the distribution, and in particular the mean, the variance and the dependence between observations a given number of steps apart, stays the same wherever you look in time.

Differencing is the tool for getting there — “compute the differences between consecutive observations” — and its division of labor with transformation is precise: “transformations such as logarithms can help to stabilize the variance of a time series,” while “differencing can help stabilize the mean of a time series by removing changes in the level of a time series, and therefore eliminating (or reducing) trend and seasonality.” The diagnostic is the ACF: “for a stationary time series, the ACF will drop to zero relatively quickly, while the ACF of non-stationary data decreases slowly.”

Familiar methods are special cases

The framework is wider than it looks, and seeing the simple methods inside it is the fastest way to build intuition.

Method	ARIMA form
White noise	ARIMA(0,0,0) with no constant
Random walk	ARIMA(0,1,0) with no constant
Random walk with drift	ARIMA(0,1,0) with a constant
Autoregression	ARIMA(\(p\),0,0)
Moving average	ARIMA(0,0,\(q\))

The random walk row explains a baseline you have already met. When a differenced series is white noise the model is \(y_t = y_{t-1} + \varepsilon_t\), and “the forecasts from a random walk model are equal to the last observation, as future movements are unpredictable” — so “the random walk model underpins naïve forecasts.” See naïve baseline forecast. Such series also “typically have long periods of apparent trends up or down” and “sudden and unpredictable changes in direction,” which is worth remembering before reading a trend into a price chart.

What d does to your long-horizon uncertainty

This is the practically decisive consequence of the specification, and it is stated directly: “for stationary models (i.e., with \(d=0\)) they will converge, so that prediction intervals for long horizons are all essentially the same. For \(d \ge 1\), the prediction intervals will continue to grow into the future.“

So the differencing order, chosen for statistical reasons, determines whether your twelve-month-out uncertainty looks like your one-month-out uncertainty or many times larger. Anyone reading the forecast should know which regime they are in — see prediction intervals.

And the intervals tend to be too narrow in both regimes. The standard calculation accounts only for the variation of the errors. It leaves out the uncertainty in the estimated parameters and in the choice of model order, and it assumes that the historical patterns the model captured will continue into the forecast period. That last assumption is the one that matters for business forecasting: a structural break can violate it, in which case future observations may fall inside or outside the reported bounds, and nominal coverage is not guaranteed once the assumed model no longer describes the process.

The interval machinery also rests on assumptions you are told to check rather than assume: “the residuals are uncorrelated and normally distributed. If either of these assumptions does not hold, then the prediction intervals may be incorrect. For this reason, always plot the ACF and histogram of the residuals to check the assumptions before producing prediction intervals.”

Before reaching for it

Order selection is genuinely hard — “selecting appropriate values for \(p\), \(d\) and \(q\) can be difficult” — and automatic selection is available and widely used. Using it responsibly means still looking at the chosen \(d\) (for the interval behavior above) and at the residual ACF (for whether the fit is adequate).

Three questions are worth asking before choosing this family at all. Does a simple benchmark already do well enough — compare against forecasting benchmarks before accepting the complexity. Do you have external drivers that matter, in which case a model with covariates fits the problem better. And is the history long and stable enough for the fitted pattern to be worth projecting, since that is the assumption the intervals depend on. See time series forecasting for the wider picture and Prophet for a different modeling philosophy.

References

• Hyndman & Athanasopoulos, Forecasting: Principles and Practice (3rd ed.), §2.8 “Autocorrelation”
• §9.1 “Stationarity and differencing”
• §9.5 “Non-seasonal ARIMA models”
• §9.8 “Forecasting”