1. Model Setup
We observe $n$ independent data points:
$y_i \mid μ, σ^2 \sim N(μ, σ^2)$
where both $μ$ (mean) and $σ^2$ (variance) are unknown.
Instead of using a noninformative prior, we now assume a conjugate prior that leads to analytically convenient posterior updates.
2. The Conjugate Prior Family
The conjugate prior for $(μ, σ^2)$ in the normal model must have the form:
$p(μ, σ^2) = p(μ \mid σ^2)\, p(σ^2)$
where:
$μ \mid σ^2 \sim N(μ_0, \frac{σ^2}{κ_0}), \quad σ^2 \sim \text{Inv-}\chi^2(ν_0, σ_0^2)$
This means:
- $μ$ given $σ^2$ is normal,
- $σ^2$ marginally is scaled inverse-χ².
This pair defines the Normal–Inverse–χ² (N–Inv–χ²) prior family, parameterized by:
$p(μ, σ^2) \propto (σ^2)^{-(ν_0/2 + 1)} \exp\!\left[-\frac{1}{2σ^2}\left(ν_0 σ_0^2 + κ_0 (μ – μ_0)^2\right)\right]$ (3.6)
We denote this joint prior as:
$(μ, σ^2) \sim \text{N-Inv-}\chi^2(μ_0, \frac{σ_0^2}{κ_0}; ν_0, σ_0^2)$
3. Interpretation of the Prior Parameters
| Parameter | Meaning |
|---|---|
| $μ_0$ | Prior mean (belief about true mean before data) |
| $κ_0$ | Prior strength for $μ$ (like an equivalent sample size) |
| $ν_0$ | Prior degrees of freedom (controls certainty about $σ^2$) |
| $σ_0^2$ | Prior scale (roughly the expected value of $σ^2$) |
4. Dependence Between $μ$ and $σ^2$
Because $μ \mid σ^2$ depends on $σ^2$, the joint prior couples them:
if $σ^2$ is large, the prior variance of $μ$ also increases.
This dependence is intuitive:
- If the data-generating process has large measurement variability ($σ^2$ large),
then uncertainty about the mean $μ$ should also be larger. - $κ_0$ can be interpreted as the number of prior equivalent observations.
5. Posterior Distribution
Using Bayes’ rule, we multiply the prior by the likelihood:
$p(μ, σ^2 \mid y) \propto p(y \mid μ, σ^2)\, p(μ, σ^2)$
The likelihood (as before) is:
$p(y \mid μ, σ^2) \propto (σ^2)^{-n/2} \exp\!\left[-\frac{1}{2σ^2}((n-1)s^2 + n(\bar{y} – μ)^2)\right]$
Multiplying this with the prior (3.6), we get:
$p(μ, σ^2 \mid y) \propto (σ^2)^{-(ν_0/2 + n/2 + 1)} \exp\!\left[-\frac{1}{2σ^2} \big(ν_0 σ_0^2 + κ_0 (μ – μ_0)^2 + (n-1)s^2 + n(\bar{y} – μ)^2\big)\right]$
After simplification, this posterior belongs to the same family:
$(μ, σ^2) \mid y \sim \text{N-Inv-}\chi^2(μ_n, \frac{σ_n^2}{κ_n}; ν_n, σ_n^2)$
6. Posterior Parameter Updates
The updated (posterior) hyperparameters are:
$\begin{aligned} κ_n &= κ_0 + n, \\[6pt] ν_n &= ν_0 + n, \\[6pt] μ_n &= \frac{κ_0 μ_0 + n \bar{y}}{κ_0 + n}, \\[6pt] ν_n σ_n^2 &= ν_0 σ_0^2 + (n-1)s^2 + \frac{κ_0 n}{κ_0 + n} (\bar{y} – μ_0)^2. \end{aligned}$
7. Interpretation of Posterior Parameters
- Posterior mean (μₙ):
Weighted average of prior mean $μ_0$ and data mean $\bar{y}$:- $μ_n = \frac{κ_0 μ_0 + n \bar{y}}{κ_0 + n}.$ The larger $κ_0$, the more influence the prior mean has.
- Posterior degrees of freedom (νₙ):
Equal to $ν_0 + n$, combining prior and data “information.” - Posterior scale (σₙ²):
Combines prior scale, sample variance, and disagreement between data and prior mean.
8. Conditional Posterior for $μ \mid σ^2, y$
Given $σ^2$, the conditional posterior of $μ$ is:
$μ \mid σ^2, y \sim N\!\left(μ_n, \frac{σ^2}{κ_n}\right)$
Explicitly:
$μ_n = \frac{\frac{κ_0}{σ^2} μ_0 + \frac{n}{σ^2} \bar{y}}{\frac{κ_0}{σ^2} + \frac{n}{σ^2}}, \quad \text{Var}(μ \mid σ^2, y) = \frac{σ^2}{κ_0 + n}$
This matches the known-variance Bayesian updating rule from earlier sections.
9. Marginal Posterior for $σ^2 \mid y$
Integrating out $μ$, we get:
$σ^2 \mid y \sim \text{Inv-}\chi^2(ν_n, σ_n^2)$
Thus, the variance posterior depends on both prior information and sample variability.
10. Sampling from the Posterior
To simulate from the joint posterior:
- Draw $σ^2$ from $\text{Inv-}\chi^2(ν_n, σ_n^2)$.
- Given $σ^2$, draw $μ$ from $N(μ_n, σ^2 / κ_n)$.
This sequential sampling mirrors the hierarchical structure of the model.
11. Marginal Posterior for μμμ
Integrating out $σ^2$ yields a Student-t distribution:
$μ \mid y \sim t_{ν_n}\!\left(μ_n, \frac{σ_n}{\sqrt{κ_n}}\right)$
That is:
$p(μ \mid y) \propto \left[1 + \frac{κ_n (μ – μ_n)^2}{ν_n σ_n^2}\right]^{-(ν_n + 1)/2}$
This distribution expresses the posterior uncertainty about the mean after fully accounting for the unknown variance.
12. Summary Table
| Quantity | Distribution | Parameters | Interpretation |
|---|---|---|---|
| $μ, σ^2$ (prior) | N–Inv–χ² | $μ_0, κ_0, ν_0, σ_0^2$ | Prior beliefs |
| $μ \mid σ^2, y$ | Normal | $μ_n, σ^2/κ_n$ | Conditional posterior |
| $σ^2 \mid y$ | Inverse-χ² | $ν_n, σ_n^2$ | Marginal posterior for variance |
| $μ \mid y$ | Student-t | $/κnν_n, μ_n, σ_n^2/κ_n$ | Marginal posterior for mean |
13. Intuitive Summary
- The conjugate prior makes Bayesian updating analytically tractable.
- Posterior parameters blend prior and data in proportion to their relative precision.
- The posterior of $μ$ remains t-distributed, just like in the noninformative case—but with updated degrees of freedom and precision reflecting the prior.
- The model links the uncertainty about $μ$ to the uncertainty about $σ^2$, a natural and realistic dependency.
14. Key Insight
The Normal–Inverse–χ² prior family provides a coherent and interpretable Bayesian framework for joint inference on both the mean and variance of normal data.
It generalizes the noninformative prior by introducing flexible parameters that represent:
- how much prior information we have ($κ_0, ν_0$),
- what we believe about the mean ($μ_0$),
- and how variable the system is ($σ_0^2$).
In short:
Noninformative prior = special case of the conjugate prior with $κ_0, ν_0 \to 0$.