1. Problem Setup
We consider $n$ independent observations $y = (y_1, y_2, \ldots, y_n)$ from a univariate normal distribution:
$y_i \mid μ, σ^2 \sim N(μ, σ^2)$
Both parameters $μ$ (mean) and $σ^2$ (variance) are unknown.
Our goal is to estimate $μ$ under a noninformative prior assumption.
2. Noninformative Prior Distribution
Assuming prior independence between location ($μ$) and scale ($σ$) parameters,
a sensible “vague” or noninformative prior is uniform on $(μ, \log σ)$.
Equivalently, the joint prior density is:
$p(μ, σ^2) \propto \frac{1}{σ^2}$
This prior is improper (does not integrate to 1), but is often used as a default baseline.
3. Joint Posterior Distribution
Applying Bayes’ theorem:
$p(μ, σ^2 \mid y) \propto p(y \mid μ, σ^2) \, p(μ, σ^2)$
The likelihood for normal data is:
$p(y \mid μ, σ^2) \propto (σ^2)^{-n/2} \exp\!\left[-\frac{1}{2σ^2} \sum_{i=1}^{n} (y_i – μ)^2 \right]$
Multiplying by the prior $1/σ^2$ gives:
$p(μ, σ^2 \mid y) \propto (σ^2)^{-(n/2 + 1)} \exp\!\left[-\frac{1}{2σ^2} \sum_{i=1}^{n} (y_i – μ)^2 \right]$
Expanding the quadratic term around the sample mean $\bar{y}$:
$\sum_{i=1}^{n} (y_i – μ)^2 = (n-1)s^2 + n(\bar{y} – μ)^2$
where $s^2 = \frac{1}{n-1} \sum_{i=1}^{n} (y_i – \bar{y})^2$
is the sample variance.
So the joint posterior becomes:
$p(μ, σ^2 \mid y) \propto (σ^2)^{-(n/2 + 1)} \exp\!\left[-\frac{1}{2σ^2}\big((n-1)s^2 + n(\bar{y} – μ)^2\big)\right]$
The sufficient statistics are $\bar{y}$ and $s^2$.
4. Conditional Posterior for $μ \mid σ^2, y$
From the previous result (known variance case),
the conditional posterior of $μ$ given $σ^2$ is:
$μ \mid σ^2, y \sim N\!\left(\bar{y}, \frac{σ^2}{n}\right)$
5. Marginal Posterior for $σ^2 \mid y$
We integrate out $μ$ from the joint posterior:
$p(σ^2 \mid y) \propto (σ^2)^{-(n/2 + 1)} \int_{-\infty}^{\infty} \exp\!\left[-\frac{1}{2σ^2}\big((n-1)s^2 + n(\bar{y} – μ)^2\big)\right] dμ$
The integral over $μ$ is Gaussian, with result proportional to $(σ^2/n)^{1/2}$.
Hence:
$p(σ^2 \mid y) \propto (σ^2)^{-(n+1)/2} \exp\!\left[-\frac{(n-1)s^2}{2σ^2}\right]$
This is the scaled inverse-χ² distribution:
$σ^2 \mid y \sim \text{Inv-}\chi^2(n-1, s^2)$
Thus, the joint posterior factors as:
$p(μ, σ^2 \mid y) = p(μ \mid σ^2, y) \, p(σ^2 \mid y)$
6. Sampling from the Joint Posterior
It is straightforward to simulate from $p(μ, σ^2 \mid y)$:
- Draw $σ^2$ from $\text{Inv-}\chi^2(n-1, s^2)$,
- Then draw $μ$ from $N(\bar{y}, σ^2/n)$.
This two-step simulation naturally respects the posterior hierarchy.
7. Marginal Posterior for $μ \mid y$
To obtain the marginal posterior for $μ$, integrate out $σ^2$:
$p(μ \mid y) = \int_0^{\infty} p(μ, σ^2 \mid y) \, dσ^2$
This results in a Student-t distribution:
$μ \mid y \sim t_{n-1}\!\left(\bar{y}, \frac{s}{\sqrt{n}}\right)$
that is, $\frac{μ – \bar{y}}{s / \sqrt{n}} \sim t_{n-1}.$
So the posterior of $μ$ coincides with the classical sampling distribution of the sample mean under unknown variance — but derived purely through Bayesian reasoning.
8. Posterior Predictive Distribution
For a future observation $\tilde{y}$, we integrate over $μ$ and $σ^2$:
$p(\tilde{y} \mid y) = \int \int p(\tilde{y} \mid μ, σ^2) \, p(μ, σ^2 \mid y) \, dμ \, dσ^2$
The result is also a t-distribution:
$\tilde{y} \mid y \sim t_{n-1}\!\left(\bar{y}, \, s \sqrt{1 + \frac{1}{n}}\right)$
To simulate:
- Draw $(μ, σ^2)$ from the joint posterior,
- Then draw $\tilde{y} \sim N(μ, σ^2)$.
9. Example — Estimating the Speed of Light
In 1882, Simon Newcomb measured the time for light to travel 7442 meters, obtaining $n = 66$observations.
- Sample mean: $\bar{y} = 26.2$
- Sample standard deviation: $s = 10.8$
Assuming the noninformative prior $p(μ, σ^2) \propto 1/σ^2$:
Posterior for $μ$: $μ \mid y \sim t_{65}\!\left(26.2, \frac{10.8}{\sqrt{66}}\right).$
A 95% central posterior interval:
$26.2 \pm 1.997 \times \frac{10.8}{\sqrt{66}} = [23.6, 28.8]$
Simulation confirmation:
- Draw $σ^2 = 65 s^2 / \chi^2_{65}$,
- Then draw $μ \sim N(26.2, σ^2 / 66)$.
The posterior median and credible interval from simulation closely match the analytic result.
However, the modern accepted value for this experiment corresponds to $μ = 33.0$, which lies outside the 95% posterior interval—showing that Bayesian inference is only as good as the model and data.
10. Summary Table
| Quantity | Posterior Distribution | Notes |
|---|---|---|
| $p(μ, σ^2 \mid y)$ | $(σ^2)^{-(n/2+1)} \exp[-((n-1)s^2 + n(\bar{y}-μ)^2)/(2σ^2)]$ | Joint posterior |
| $μ \mid σ^2, y$ | $N(\bar{y}, σ^2/n)$ | Conditional |
| $σ^2 \mid y$ | $\text{Inv-}\chi^2(n-1, s^2)$ | Marginal |
| $μ \mid y$ | $t_{n-1}(\bar{y}, s/\sqrt{n})$ | Marginal posterior for mean |
| $\tilde{y} \mid y$ | $t_{n-1}(\bar{y}, s\sqrt{1 + 1/n})$ | Posterior predictive |
11. Intuitive Summary
- The noninformative prior $p(μ, σ^2) \propto 1/σ^2$ leads to standard Bayesian results identical to classical inference for normal data.
- Posterior of $μ$ is a t-distribution with $n-1$ degrees of freedom.
- Posterior of $σ^2$ is a scaled inverse-χ² distribution.
- Predictive distribution for a future observation is also t-distributed.
- The method demonstrates how Bayesian marginalization automatically accounts for nuisance parameters (σ²).
In short:
With a flat prior on the mean and scale, the Bayesian posterior inference for normal data reproduces the familiar t-based results from classical statistics—but now grounded in probability statements about parameters themselves.