After MCMC chains have mixed and converged, we want to know:

How many independent draws are these correlated MCMC samples equivalent to?

This is called the effective sample size, written as $\hat{n}_{\text{eff}}$​.

Even if we have thousands of saved MCMC draws, these draws are correlated, so the real amount of independent information may be much smaller.

1. Why do we need effective sample size?

Imagine:

  • We run m chains.
  • Each chain contributes n post-warm-up iterations.
  • Total draws saved: mnm nmn.

If all draws were independent, the between-chain variance $B$ would estimate $\text{Var}(ψ \mid y)$, and $mn$ would be the true sample size.

But MCMC draws within each chain are autocorrelated — sometimes heavily.
Thus:

  • Within-chain variance $W$ is too small
  • Between-chain variance $B$ is too big
  • The result: effective sample size < total sample size

2. Definition of ESS using asymptotic autocorrelation theory

Suppose: $ψ^{(1)}, ψ^{(2)}, …, ψ^{(n)}$

is a correlated MCMC sequence for a scalar estimand $ψ$.

The classical asymptotic variance formula for the mean of a correlated sequence is:

$\lim_{n\to\infty} \left[ m n \; \text{Var}(\bar{ψ}) \right] = \text{Var}(ψ\mid y)\left(1 + 2\sum_{t=1}^\infty \rho_t\right)$

Here:

  • $ρ_t$​ = autocorrelation of the chain at lag $t$
  • The sum $1 + 2\sum\rho_t$​ is called the autocorrelation inflation factor.

This inflates the variance because correlated chains contain less independent information.

Therefore:

$\hat{n}_{\text{eff}} = \frac{m n}{1 + 2\sum_{t=1}^\infty \rho_t}$

This answers:

“How many independent draws would give the same precision as my correlated MCMC draws?”

3. How do we estimate the autocorrelations $ρ_t$​?

We need an empirical estimate of $\sum ρ_t$​.
But directly estimating high-lag correlations is noisy.

So the authors use the variogram:

$V_t = \frac{1}{ m (n – t) } \sum_{j=1}^m \sum_{i=t+1}^{n} (ψ_{i,j} – ψ_{i-t,j})^2$

Interpretation:

  • It measures average squared differences between draws $t$ apart.
  • For a stationary process:
    • $E[(ψ_i – ψ_{i-t})^2] = 2(1 – ρ_t)\,\text{Var}(ψ)$

Thus:

$\hat{ρ}_t = 1 – \frac{V_t}{2\,\text{Var}^+(ψ\mid y)}$

where:

  • $\text{Var}^+$ = combined (overdispersed) variance estimator
  • This adjusts for chain-to-chain and within-chain variability.

4. When to stop summing correlations (truncation rule)

We cannot sum to infinity.
High-lag correlations are too noisy.

So we stop early using the Geyer/Brooks–Gelman approach:

Find the first odd lag T where:

$\hat{ρ}_{T+1} + \hat{ρ}_{T+2} < 0$

Then stop summing at lag T.

This guarantees the total sum remains positive and stable.

5. Final ESS estimator

Putting it all together:

$\hat{n}_{\text{eff}} = \frac{m n}{ 1 + 2\sum_{t=1}^{T} \hat{ρ}_t }$

Where:

  • $m n$ = total number of post-warm-up draws
  • $T$ = first odd integer where $\hat{ρ}_{T+1} + \hat{ρ}_{T+2} < 0$

This $\hat{n}_{\text{eff}}$​ is the number of independent draws the MCMC sample is equivalent to.

6. Important practical notes

(a) Only use post–warm-up iterations

Compute ESS using only draws after discarding warm-up.

Example:

  • 4 chains × 1000 iterations each = 4000 draws
  • Warm-up = half → keep 2000
  • Split each chain → m = 8 chains, each n = 250
  • ESS computed from these 2000 kept draws only.

(b) Transform parameters with bounded or skewed posteriors

The ESS formula assumes near-normal behavior.

If a scalar $ψ$ has a:

  • bounded distribution (0–1) → use logit transform
  • positive-only distribution → use log transform
  • very long-tailed distribution → use rank transform

This helps make mean-variance diagnostics more reliable.

7. Using ESS and $\hat{R}$ to decide when to stop simulations

You must check both:

  1. Potential scale reduction factor $\hat{R}$
    • Convergence diagnostic
    • Want $\hat{R} \approx 1$
  2. Effective sample size $\hat{n}_{\text{eff}}$
    • Precision diagnostic
    • Want enough independent information

Recommended stopping rule:

Run simulation until:

$\hat{n}_{\text{eff}} \ge 5m$

(which corresponds to ≈10 independent draws per original chain)

This ensures stability across chains.

8. Key caution: Even passing diagnostics does not guarantee full convergence

Both $\hat{R}$ and ESS rely on:

  • chains mixing across the explored region
  • chains being locally stationary

But:

If the chain never reached certain parts of the true posterior distribution
(e.g., due to multimodality, bad starting points, bad proposal),
these diagnostics WILL NOT detect the problem.

Thus, convergence diagnostics evaluate:

“Have the observed chains mixed with each other?”

They do not prove:

“Have the chains explored the entire true posterior space?”

This is why model understanding, reparameterization, and careful checking remain critical.

Final Summary (Short Version)

  • ESS measures how many independent draws your correlated MCMC sample is equivalent to.
  • It is computed using autocorrelations estimated via variograms.
  • The sum of autocorrelations is truncated when adjacent terms turn negative.
  • The formula is:
    • $\hat{n}_{\text{eff}} = \frac{m n}{1 + 2\sum_{t=1}^{T} \hat{ρ}_t}$
  • Use only post-warm-up draws.
  • For skewed or bounded parameters, transform before computing ESS.
  • Stop running MCMC when:
    • $\hat{R} \approx 1$, and
    • $\hat{n}_{\text{eff}}$​ ≳ 101010 per chain.