Overdispersed versions of standard models

Standard distributions such as the binomial, normal, Poisson, and exponential often fail when observed data exhibit overdispersion. Overdispersion occurs when variability in the data exceeds the variance predicted by the standard model. A normal model, for example, is inappropriate for large samples where more than about 10% of observations fall beyond $1.5$ times the interquartile range. A natural extension for each standard model introduces an additional parameter to allow the variance to exceed the value imposed by the baseline distribution. Every such extension can be written as a mixture distribution.

A common feature of these extended families is that underdispersion is impossible. The mean remains the same as in the standard model, but the variance increases, consistent with mixture interpretations in which the overall mean equals the average of the conditional means while the overall variance includes both the average conditional variance and the additional variability arising from differences in the conditional means.

---

1. $t$ Distribution as an Alternative to the Normal

The $t_\nu(\mu, \sigma^2)$ distribution provides robustness by allowing longer tails than the normal.
Key properties:

• Parameters: location $\mu$, scale $\sigma$, and degrees of freedom $\nu$.
• When $\nu = 1$, the distribution becomes the Cauchy, with infinite mean and variance.
• As $\nu \to \infty$, the distribution approaches the normal.
• When the $t$ is used to model genuinely heavy-tailed data, $\nu$ should be treated as an unknown parameter.
• When used purely as a robust alternative, $\nu$ can be fixed at a small value, but not so small that the implied variance becomes unrealistic.

Mixture Representation

A $t$ model for $y_i$ can be written as:

• $y_i \mid V_i \rightarrow N(\mu, V_i)$
• $V_i \rightarrow \text{Inv-}\chi^2(\nu, \sigma^2)$

This representation identifies observations with large $V_i$ as natural outliers. A similar interpretation applies to exchangeable parameters such as $\theta_j$.

---

2. Negative Binomial Alternative to Poisson

The Poisson distribution requires variance equal to the mean. Many count datasets violate this condition, exhibiting variance greater than the mean.

The negative binomial distribution addresses this problem.

• Let $y_1, \dots, y_n \sim \text{Neg-bin}(\alpha, \beta)$.
• This distribution is equivalent to:
  • $y_i \mid \lambda_i \rightarrow \text{Poisson}(\lambda_i)$
  • $\lambda_i \rightarrow \text{Gamma}(\alpha, \beta)$

Mean: $\alpha / \beta$
Variance: $(\beta + 1)/\beta \cdot \alpha / \beta$, which always exceeds the mean.

As $\beta \rightarrow \infty$ while $\alpha/\beta$ stays constant, the gamma distribution becomes degenerate at its mean, and the negative binomial converges to the Poisson.

---

3. Beta-Binomial Alternative to Binomial

The binomial distribution has a single free parameter, so variance is determined by the mean. The beta-binomial adds flexibility.

• Data $y_i$ are modeled as $\text{Beta-bin}(m, \alpha, \beta)$.
• Interpretation:
  • $y_i \mid \pi_i \rightarrow \text{Binomial}(m, \pi_i)$
  • $\pi_i \rightarrow \text{Beta}(\alpha, \beta)$

Mean probability: $\alpha / (\alpha + \beta)$.

The variance is inflated by a factor of $(\alpha + \beta + m)/(\alpha + \beta + 1)$ compared to the binomial.
If $m = 1$, then the beta-binomial and binomial have identical variances because there is no information to distinguish the two variation sources.

---

4. $t$ Distribution Alternative for Logistic and Probit Regression (“Robit Regression”)

In logistic and probit regression, large values of the linear predictor $X\beta$ map to probabilities extremely close to $0$ or $1$. Robustness issues arise from predictors rather than $y_i$, because binary responses do not reveal tail behavior.

A robust approach replaces the logistic or normal latent distribution with a $t$ distribution:

• Latent variable formulation:
  $u_i \rightarrow t_\nu((X\beta)_i, 1)$.

The parameter $\nu$ is typically fixed at a small value (for example, $\nu = 4$) because $u_i$ is unobserved and cannot support inference for $\nu$. As $\nu \to \infty$, the model approaches the probit.

Computation uses the EM algorithm or a Gibbs sampler, applying the normal-mixture representation for the $t$ distribution. Each $u_i$ and its variance are treated as missing data.

---

5. Why Use Standard (Nonrobust) Models at All?

Although robust alternatives exist, standard models remain useful:

• The binomial and multinomial apply naturally to repeated independent trials with fixed totals.
• The Poisson and exponential arise directly from Poisson processes.
• The normal fits data that aggregate many independent components because of the central limit theorem.

In the educational testing example from previous section, each observed effect $y_j$ is an average of approximately $60$ student-level adjusted test scores. The sampling distribution of $y_j$ is well approximated by
$y_j \mid \theta_j, \sigma_j^2 \rightarrow N(\theta_j, \sigma_j^2)$.

Standard models are also computationally convenient:

• Conjugate priors provide closed-form posterior means and variances.
• Simulation and inference are simpler.

When a standard model is chosen for convenience, it is advisable to:

1. Check model fit via the posterior predictive distribution.
2. Perform a sensitivity analysis by comparing posterior inferences under extended families (for example, replacing a normal with a $t$).