A hierarchical linear model with multiple levels can be transformed into a single enlarged regression model when all distributions involved are normal. This transformation relies on normal–normal conjugacy, which makes it possible to reinterpret prior and hyperprior distributions as additional data points.

1. Three-level hierarchical structure

A general three-level model is written as:

Data level (likelihood)

$y \mid X, \beta, \Sigma_y \sim \text{Normal}(X\beta, \Sigma_y)$

  • $y$: vector with $n$ observed outcomes
  • $X$: design matrix for the data
  • $\beta$: regression coefficients
  • $\Sigma_y$​: covariance matrix for observational noise

Group-level distribution (population model)

$\beta \mid X_\beta, \alpha, \Sigma_\beta \sim \text{Normal}(X_\beta \alpha, \Sigma_\beta)$

  • $\beta$: $J$ group-level parameters
  • $X_\beta$​: predictors describing variation among the $\beta$ terms
  • $\alpha$: hyperparameters
  • $\Sigma_\beta$​: covariance controlling how far β values vary around $X_\beta \alpha$

Hyperprior level

$\alpha \mid \alpha_0, \Sigma_\alpha \sim \text{Normal}(\alpha_0, \Sigma_\alpha)$

  • $\alpha$: $K$ hyperparameters
  • $\alpha_0$​: hyperprior mean
  • $\Sigma_\alpha$​: hyperprior covariance

This structure defines three connected layers:

  1. observed data, 2) group-level parameters, 3) hyperparameters.

2. Conjugacy → all levels can be combined into one regression

Normal conjugacy allows the entire hierarchy to be represented as:

$y_* \mid x_*, \gamma, \Sigma_* \sim \text{Normal}(x_* \gamma, \Sigma_*)$

where:

  • $\gamma = (\beta, \alpha)$: parameter vector of length $J + K$
  • $y_*$​: augmented response vector
  • $x_*$​: augmented design matrix
  • $\Sigma_*$​: augmented covariance or precision matrix

In this formulation, likelihood information, prior information about β, and hyperprior information about α all appear as if they were additional observations.

3. Explicit construction of $y_*, x_*, \Sigma_*^{-1}$

Augmented response vector

y$y_* = \begin{pmatrix} y \\ 0 \\ \alpha_0 \end{pmatrix}$

Interpretation:

  • The first block is the actual observed data $y$.
  • The second block (zeros) corresponds to the prior relationship
    • $\beta – X_\beta \alpha \sim \text{Normal}(0, \Sigma_\beta)$
  • The final block corresponds to $\alpha \sim \text{Normal}(\alpha_0, \Sigma_\alpha)$

Total length:

$n + J + K$

Augmented design matrix

$x_* = \begin{pmatrix} X & 0 \\ I_J & -X_\beta \\ 0 & I_K \end{pmatrix}$

Block-by-block meaning:

  • Upper block: usual data regression, connecting $y$ to $\beta$.
  • Middle block: expresses $\beta – X_\beta \alpha = 0$ in linear form.
  • Lower block: extracts α for hyperprior information.

Augmented precision matrix

$\Sigma_*^{-1} = \begin{pmatrix} \Sigma_y^{-1} & 0 & 0 \\ 0 & \Sigma_\beta^{-1} & 0 \\ 0 & 0 & \Sigma_\alpha^{-1} \end{pmatrix}$

Each diagonal block adds information from one of the three hierarchical levels.

4. Handling noninformative components

A component of $\beta$ or $\alpha$ may have a noninformative prior, which corresponds to infinite variance (zero precision).
Any row associated with such a component is removed from:

  • $y_*$
  • $x_*$
  • $\Sigma_*^{-1}$

To formalize this:

  • $J_*$: number of β components with informative priors
  • $K_*$: number of α components with informative priors

The effective enlarged regression includes:

$n + J_* + K_*$

observations.

5. Election forecasting illustration

In one election forecasting model:

  • The vector $\beta$ contains 75 coefficients:
    • 20 baseline predictors
    • 11 year-specific error terms
    • 44 region × year coefficients

Only 55 coefficients have informative priors:

$J_* = 11 + 44 = 55$

All means for the varying coefficients are fixed constants rather than random, so:

$K_* = 0$

Thus the augmented regression incorporates:

$n + 55$

observations.

6. Multiple equivalent hierarchical representations

A hierarchical model often has several mathematically equivalent forms.

8-schools example

Two common versions:

$y_j \sim \text{Normal}(\theta_j, \sigma_j^2), \quad \theta_j \sim \text{Normal}(\mu, \tau^2)$

$y_j \sim \text{Normal}(\beta_0 + \beta_j, \sigma_j^2), \quad \beta_j \sim \text{Normal}(0, \tau^2)$

Both versions generate identical probabilistic behavior.

7. Different valid formulations of the election forecasting model

Two approaches lead to the same structure:

  1. A regression with 70 predictors and normal priors on selected coefficients.
  2. A regression with 20 predictors, plus three random-effect components:
    • year effects
    • region × year effects
    • state × year effects

Each approach describes the same underlying hierarchical pattern.

8. Placement of predictors

Predictors for group-level variation can appear in:

  • the likelihood level,
  • the population-level distribution for β, or
  • any higher-level structure.

The intercept can also be assigned to any one of the three levels.
All choices are mathematically legitimate; the preferred form depends on convenience and interpretability.