Computing ACF’s of causal AR(2) processes using difference equations

Big picture: what you are trying to compute

When you have a time-ordered data series $X_t$​ (today, yesterday, two days ago, …), a common question is:

• How strongly is $X_t$​ related to its past values $X_{t-1}, X_{t-2}, \dots$?

That relationship is summarized by the autocorrelation function (ACF):

• $\rho(h)$ = correlation between $X_t$ and $X_{t-h}$
• $h$ is the lag (how many time steps back)

Key facts:

• $\rho(0)=1$ (a value is perfectly correlated with itself)
• If $\rho(h)$ is large in magnitude for some $h$, the series has “memory” at that lag.
• If $\rho(h)$ quickly decays to 0, the series forgets quickly.

---

The model: AR(1) and AR(2) in plain words

White noise $W_t$

Think of $W_t$ as the completely unpredictable “new shock” at time $t$.
It has mean 0 and no correlation over time.

AR(1)

$$X_t – \phi X_{t-1} = W_t \quad\Longleftrightarrow\quad X_t = \phi X_{t-1} + W_t$$

Interpretation:

• Today is a fraction $\phi$ of yesterday, plus a new shock.

AR(2)

$$X_t – \phi_1 X_{t-1} – \phi_2 X_{t-2} = W_t \quad\Longleftrightarrow\quad X_t = \phi_1 X_{t-1} + \phi_2 X_{t-2} + W_t$$

Interpretation:

• Today depends on the last two values, plus a new shock.

---

Why “causal” matters (in everyday terms)

A model is causal if $X_t$ can be written using only today and earlier shocks:$$X_t = W_t + \psi_1 W_{t-1} + \psi_2 W_{t-2} + \cdots$$

So you do not need future information to define the present.

This is crucial for forecasting: a model that needs future shocks is not usable as a practical forward-looking model.

---

Two ways to get the ACF — and why this method is simpler

A standard method is:

1. Rewrite $X_t$​ as an infinite sum of past shocks ($\psi$-weights),
2. Compute covariances from those weights.

That can be messy for anything beyond AR(1).

The alternative method here is cleaner:

Use the model equation itself to create a difference equation for $\rho(h)$

A “difference equation” is like a recurrence rule:

• it tells you how $\rho(h)$ relates to earlier values like $\rho(h-1)$, $\rho(h-2)$, etc.

Once you have that recurrence and a couple of starting values, you can compute the entire ACF.

---

Part A. AR(1): ACF via a difference equation

Start with:$$X_t – \phi X_{t-1} = W_t$$

Multiply both sides by $X_{t-h}$​ and take expectations.

For $h\ge1$, the key idea is:

• $W_t$​ is independent of the past, and if the model is causal then $X_{t-h}$​ depends only on shocks at time $t-h$ and earlier.
• So $E(W_t X_{t-h})=0$ for $h\ge1$.

That gives:$$\gamma(h) – \phi \gamma(h-1) = 0 \quad (h\ge1)$$

Divide by $\gamma(0)$ to convert to ACF:$$\rho(h) – \phi \rho(h-1)=0,\quad \rho(0)=1$$

So:$$\rho(h)=\phi^h \quad (h\ge0)$$

and by symmetry of correlation:$$\rho(h)=\phi^{|h|}$$

Meaning: AR(1) correlations decay geometrically.

---

Part B. AR(2): difference equation for the ACF

Start with:$$X_t – \phi_1 X_{t-1} – \phi_2 X_{t-2} = W_t$$

Do the same trick (multiply by $X_{t-h}$, take expectations). For $h\ge1$, again $E(W_t X_{t-h})=0$ under causality, so you get:$$\gamma(h)-\phi_1\gamma(h-1)-\phi_2\gamma(h-2)=0$$

Divide by $\gamma(0)$:$$\rho(h)-\phi_1\rho(h-1)-\phi_2\rho(h-2)=0,\quad (h\ge1)$$

This is a second-order recurrence: each value depends on the previous two.

So if you know:

• $\rho(0)=1$
• $\rho(1)=?$

then you can compute $\rho(2)$, then $\rho(3)$, and so on.

---

How to get $\rho(1)$ for AR(2)

Use the same recurrence at $h=1$:$$\rho(1)-\phi_1\rho(0)-\phi_2\rho(-1)=0$$

And use the symmetry $\rho(-1)=\rho(1)$, plus $\rho(0)=1$:$$\rho(1)-\phi_1-\phi_2\rho(1)=0$$$$(1-\phi_2)\rho(1)=\phi_1$$$$\rho(1)=\frac{\phi_1}{1-\phi_2}$$

Now you have the two starting points, and the recurrence generates all lags.

---

Closed-form “shape” of the ACF for AR(2)

To understand what $\rho(h)$ looks like, you connect the recurrence to the roots of a polynomial.

Define the AR polynomial:$$\phi(z)=1-\phi_1 z-\phi_2 z^2$$

Its roots $r_1, r_2$​ determine the form of the solution.

Case 1: Two distinct real roots

If $r_1\neq r_2$​, then:$$\rho(h)=c_1 r_1^{-h}+c_2 r_2^{-h},\quad (h\ge0)$$

This means the ACF is a sum of two decaying exponentials, possibly with sign changes depending on the roots.

Case 2: One repeated real root

If $r_1=r_2=r_0$​, then:$$\rho(h)=(c_1+c_2 h) r_0^{-h},\quad (h\ge0)$$

So it’s still decaying, but the extra “$h$” term can change the curvature.

Case 3: Complex conjugate roots

If the roots are complex conjugates, the ACF has the form:$$\rho(h)=K\cdot \beta^h \cos(h\Phi+\Theta)$$

That is:

• $\beta^h$ gives exponential decay
• $\cos(\cdot)$ gives oscillation (alternating positive/negative in a wave pattern)

This explains why some AR(2) processes have an ACF that “wiggles” around zero while shrinking.

---

Why roots outside the unit circle means “causal”

The causality condition for AR(2) is:

• both roots $r_1, r_2$​ satisfy $|r_1|>1$ and $|r_2|>1$

Practical meaning:

• the influence of past shocks fades over time instead of exploding
• the process can be represented using past shocks only (no future terms needed)

---

What the examples are illustrating

Example 3.1 (real distinct roots, negative inverses)

You get something like:$$\rho(h)=c_1(-0.4)^h+c_2(-0.9)^h$$

Because the bases are negative, the sign flips as $h$ increases:

• ACF alternates +/− and decays.

Example 3.2 (repeated root)

You get:$$\rho(h)=(1+0.83h)\cdot 0.3^h$$

This is:

• always positive (since $0.3^h>0$)
• decays very fast, but the $(1+0.83h)$ factor changes the early-lag shape.

Example 3.3 (complex roots)

You get:$$\rho(h)=K\beta^h\cos(h\Phi+\Theta)$$

So the ACF:

• oscillates (wave pattern)
• shrinks over time (because $\beta^h$ decays)

---

The practical “recipe” to compute an AR(2) ACF

1. Check causality
   Find the roots of $\phi(z)=1-\phi_1 z-\phi_2 z^2$.
   If both roots have magnitude $>1$, it is causal.
2. Write the recurrence $$\rho(h)=\phi_1\rho(h-1)+\phi_2\rho(h-2)$$
3. Set initial values
   • $\rho(0)=1$
   • $\rho(1)=\phi_1/(1-\phi_2)$
4. Either
   • compute $\rho(h)$ recursively for as many lags as you want, or
   • use the closed-form formula based on the root type (distinct, repeated, complex)

---

Why this method is valuable

• You avoid computing an infinite series representation of the process.
• You get a direct recurrence rule for the ACF.
• The root structure immediately explains the qualitative shape:
  • monotone decay,
  • alternating decay,
  • oscillatory decay.