Prior Belief (or Prior Probability)

A prior is what a Bayesian analysis assumes about a parameter before the data arrives. It is not a mood or a hunch in the final arithmetic — it is a distribution, and it enters Bayes' theorem as a factor with a measurable weight.

The prior should be stated in a form that can be checked. Its influence may be easy to quantify in a conjugate model, while more complex models require sensitivity and predictive checks rather than one universal prior-weight number.

How much of the answer the prior is

For a coin’s unknown head probability θ, a Beta(a,b) prior has density proportional to θ^(a−1)(1−θ)^(b−1), for a,b positive. Its mean is a/(a+b). After h heads and t tails under a conditionally independent, constant-probability model, the posterior is Beta(a+h,b+t): the data adds its exponents to those already present in the prior. That conjugate update makes prior influence visible.

Beta(1,1) and Beta(20,20) both have mean 0.5, but the second is much more concentrated around it. With seven heads and three tails, their posterior means are 8/12 = 0.6667 and 27/50 = 0.54. With 700 heads and 300 tails, the corresponding means are 701/1002 ≈ 0.6996 and 720/1040 ≈ 0.6923. The difference shrinks in this fixed-prior example as data accumulates, but the finite-data answer still reflects the prior.

The posterior mean (a+h)/(a+b+n), with n = h+t, can be rewritten as [(a+b)/(a+b+n)] × [a/(a+b)] + [n/(a+b+n)] × [h/n] when n is positive. The first term weights the prior mean and the second weights the observed proportion. This identity explains the pull toward 0.5 without needing to run an experiment.

For this Beta-binomial posterior mean, \(a+b\) is an effective sample-size weight: the prior mean and sample proportion have weights \(a+b\) and \(n\). Thus Beta(20,20) contributes 80% of the averaging weight at ten observations, 50% at forty, and 3.8% at a thousand. This describes a particular summary, not a literal record of imaginary experiments.

Other summaries lead to different pseudo-count conventions. The Beta density has exponents \(a-1\) and \(b-1\), which appear in the interior mode formula when both posterior shape parameters exceed one. State which interpretation you use, then defend the prior using available information or a regularization purpose and assess sensitivity.

The three kinds, and what each is for

• An informative prior encodes real prior knowledge — previous trials, a physical constraint, an established base rate. Its relevance depends on whether that information applies to the present model and population.
• A weakly informative prior rules out the absurd without asserting much: for example, excluding implausibly large effects on a specified measurement scale. What counts as weak depends on the units, likelihood, and domain.
• A flat prior is uniform in the chosen parameterization. Beta(1,1) has constant density over a proportion: it leaves the likelihood shape unchanged, although its posterior-mean formula has an effective weight of two.

The flat case carries a subtlety worth knowing before relying on it. Flat is relative to a parameterization: a prior that is uniform on a proportion is not uniform on the log-odds of that proportion, and the two give different answers. “I used a non-informative prior” is therefore not a complete description of what was assumed, only a claim that the assumption was meant to be mild.

Reporting it, and testing it

Before fitting, examine the observations implied by the prior together with the likelihood. A prior that looks broad on a parameter scale can imply mostly impossible outcomes after a nonlinear transformation. Prior predictive checks ask whether those implied outcomes are plausible. Historical observations can inform a prior, but reusing the same observations as new likelihood data would count them twice unless the joint model explicitly accounts for that use.

Report the distribution, its parameters and the reasons for choosing it. For example, Beta(20,20) has weight forty in this posterior-mean calculation; that does not say forty historical tosses were observed. Rerun the analysis under plausible alternative priors. Stability across those alternatives is useful evidence of robustness to those choices, while a changed conclusion reveals sensitivity that the report should make visible.

More observations do not rescue every prior or model. If an entire relevant region is excluded from prior support, ordinary updating cannot place mass there; weak identification or a misspecified likelihood can also preserve sensitivity. Within a suitable identifiable model, the prior may become less influential as information grows. Sensitivity deserves particular attention where the data is thin: early in an experiment, in a small subgroup, at the tail of a distribution. Those are exactly the places where a prior is most tempting and where its weight is largest, which is why the posterior should never be quoted without it.

References

• SciPy documentation, scipy.stats.beta. docs.scipy.org
• NumPy documentation, numpy.linspace. numpy.org