This section is about how to view classical/statistical procedures through a Bayesian lens. The authors describe three levels of connection:

  1. Large-sample agreement: when $n$ is big and the model is correct, Bayesian and frequentist answers often look the same (posterior ≈\approx≈ MLE ± SE).
  2. Implicit priors: many classical methods for small samples can be seen as approximations to Bayesian inference with some (often hidden) prior.
  3. Real differences: some classical tools (esp. hypothesis tests, multiple comparisons) can give conclusions that don’t match Bayesian logic.

Then they walk through several classical ideas and show how a Bayesian would think about them.

1. Using classical procedures as rough Bayesian models

Sometimes a simple data-analytic rule corresponds exactly to Bayesian inference under a certain model.

Example: ratio estimation.
In surveys, people estimate a ratio $R = \frac{y}{x}$​ using the sample ratio $R \approx \frac{y_{\text{obs}}}{x_{\text{obs}}}$​​.
This turns out to be the Bayesian posterior summary if you assume $y_i \mid x_i \sim N(R x_i, \sigma^2 x_i)$

and use a noninformative prior on $R$.
So: if your data really don’t match that model, the ratio estimator is suspect — thinking Bayesianly helps you see when the classic method is appropriate.

Similarly, variable selection by “significance” in regression is a bit like having a spike-and-slab prior: each coefficient is either exactly 0 or drawn from a wide distribution. Making that explicit suggests better versions (e.g. grouping coefficients, hierarchical priors).

2. Maximum likelihood and other point estimates

From a Bayesian point of view:

  • For large $n$ and a regular model, the MLE $\hat{\theta}$ is basically a sufficient statistic for $\theta$.
  • With a locally flat prior, the Bayesian posterior is $p(\theta \mid \hat{\theta}) \approx N\bigl(\hat{\theta}, \, [n J(\hat{\theta})]^{-1}\bigr),$ which is the same form as classical large-sample inference.

So classical MLE + SE can be read as: “this is the Bayesian posterior under a convenient weak prior, in large samples.”

But: when $n$ is small or the number of parameters is large, just using the MLE is wasteful — it ignores structure that a hierarchical prior could borrow across parameters.

3. Unbiased estimation (and why Bayesians don’t love it)

Classically, an estimator $\hat{\theta}(y)$ is unbiased if $E(\hat{\theta}(y) \mid \theta) = \theta \quad \text{for all } \theta.$

This is attractive in frequentist theory, but Bayesians see problems:

  • Unbiasedness is a sampling-theory property; it ignores what we actually know about related parameters.
  • Enforcing unbiasedness often increases variance.
  • With several related parameters, you can’t make everything unbiased at once.

Their example (mother–daughter heights):
Posterior expectation $E(\theta \mid y) = 160 + 0.5(y – 160)$

“shrinks” toward the population mean — this is regression to the mean.
It is biased in the frequentist sense (if you condition on the daughter’s true height and resample the mother), but it is more sensible.
The “unbiased” estimator $\hat{\theta} = 160 + 2(y – 160)$ is absurd for tall mothers.
So: Bayesian shrinkage beats formal unbiasedness.

Big idea: Bayesian analysis is a systematic version of regression to the mean.

4. Confidence intervals vs Bayesian intervals

  • Often, Bayesian $(1-\alpha)$ posterior intervals have close to $(1-\alpha)$ frequentist coverage, even for moderate $n$.
  • But coverage alone does not guarantee sensible intervals — you can make a 95% CI that is empty 5% of the time and the whole real line 95% of the time; it has 95% coverage but is useless.
  • So: from a Bayesian perspective, we like intervals that actually summarize posterior uncertainty, not ones built only to satisfy a coverage property.

5. Hypothesis testing

They are pretty blunt here: in the Bayesian setup, testing a point null $\theta = \theta_0$​ is rarely natural unless you put positive prior mass on that point.

For a continuous parameter, that means a prior like $p(\theta) = \pi \, \delta_{\theta_0} + (1-\pi)\, g(\theta),$

which is often artificial.

Instead of “Is $\theta = 0$?” the Bayesian asks “What is the posterior for $\theta$?” or “How big is θ\thetaθ likely to be?”

They note: sometimes a classical one-sided p-value and a Bayesian tail probability look similar (e.g. $y \sim N(\theta, 1)$, flat prior ⇒ $\Pr(\theta>0 \mid y=1) = 0.84$, but the Bayesian statement (“84% chance it’s positive”) is clearer than “fail to reject.”

They do keep one kind of “testing”: model checking — comparing data to posterior predictive distributions.

6. Multiple comparisons → use hierarchical Bayes

Classical multiple comparison methods try to control false positives across many pairwise differences. That often leads to long lists of “not significant.”

Bayesian alternative:

model $y_j \sim N(\theta_j, \sigma^2), \quad \theta_j \sim \text{some population distribution}$

(and estimate that population distribution). Then:

  • Each $\theta_j$​ is partially pooled toward the group.
  • Big differences survive; noisy differences shrink.
  • You can compute things like $\Pr(\theta_j > \theta_k \mid y)$ directly.
  • This automatically addresses the “searching among many comparisons” problem.

So: hierarchical models solve the multiple-comparisons issue rather than correct for it afterward.

7. Nonparametric methods, permutation, jackknife, bootstrap

They say: many classical nonparametric tools are hard to interpret Bayesianly because they don’t start from a full probability model.

  • Rank tests (like Wilcoxon) can be reinterpreted as: transform data to ranks/quantiles, then do a normal-based comparison — that part can be seen as an approximate Bayesian inference on the transformed scale.
  • But once things get complicated (covariates, censoring, missingness), it’s often easier in Bayes: write down a joint model and check it.

So they don’t reject nonparametric methods — they treat them as useful descriptive tools or as ways to suggest a model. But for rich inference, they prefer a model-based Bayesian approach.

8. Big picture

  • Many classical methods are approximate Bayes with an implicit prior.
  • Large-sample classical inference is often the same as Bayes with a flat prior.
  • Where classical methods feel awkward (point nulls, multiple comparisons, complicated structures), Bayesian modeling is cleaner.
  • And when classical nonparametric tools work well (balanced experiments, simple surveys), they tend to match Bayesian analyses with noninformative priors.

So this section is basically: “You can read a lot of classical statistics as rough Bayesian inference — and seeing the implicit prior helps you know when the classical method is appropriate and when to replace it with a full model.”