Posterior Probability

A posterior probability is a single number taken out of a posterior distribution: the probability that a hypothesis is true, or that a parameter falls in some region, given the data. It is what people usually want when they ask how likely something is.

It is available because the Bayesian framework treats the hypothesis as uncertain and the data as fixed. That is what buys the direct statement, and it is also what the number costs.

The number carries its prior with it

Take an explicitly restricted model with only two hypotheses: H₀ says the coin’s head probability is 0.5 and H₁ says it is 0.7. Seven heads in ten conditionally independent tosses have binomial probability about 0.1171875 under H₀ and 0.266828 under H₁. With equal prior mass, P(H₁ | D) = 0.266828 / (0.1171875 + 0.266828) ≈ 0.6948. This is the probability of the specified H₁ within this two-point model, not a probability covering every possible form of bias. Start from a prior of 0.1 and it is 0.2019; start from 0.01 and it is 0.0225. Same data, same likelihoods, same theorem.

So a posterior probability is not a property of the data. Quoting “a 95% probability that B beats A” without the prior is quoting half a result. The convention of using an even prior is defensible, but it is a choice that should be written down rather than assumed to be neutral — it asserts that the two hypotheses were equally plausible before the experiment, which is often not what anyone believed.

Not a p-value, and not comparable to one

The same data gives a one-sided p-value of 0.1719 and a posterior probability of H₁ of 0.6948. Neither is 1 minus the other, and there is no conversion between them, because they answer different questions.

• The p-value is computed under the null model alone: if the coin were fair, how often would I see 7 or more heads? It needs no prior and no rival hypothesis, and it says nothing about the probability that the coin is fair.
• This two-hypothesis posterior calculation requires both prior probabilities and a specified alternative — here, exactly 0.7. Change the alternative to 0.6 or 0.9 and the number changes, because the likelihood under \(H_1\) changes.

The frequent claim that a posterior probability is the “intuitive version” of a p-value is therefore half right. It is intuitive, and it is a different quantity requiring two inputs the p-value never asked for. The interpretive advantage is real; it is not free.

Using it as a decision rule

A different model can place a continuous Beta(1,1) prior over all proportions between zero and one. The same seven heads and three tails then give Beta(8,4), and P(θ > 0.5 | D) is about 0.8867. This does not contradict 0.6948: the hypothesis space, prior, and queried event differ. Under this continuous posterior, P(θ = 0.5 | D) is zero, as is the probability of every other exact point. Giving an exact null positive posterior mass requires a prior model that assigns it positive mass.

For a decision, specify the event that matters before choosing a threshold. P(effect > 0 | D) differs from P(effect > δ | D), where δ is the smallest practically useful effect. Expected loss can also use the whole posterior instead of reducing it to whether one boundary is crossed. A highly probable tiny improvement may still be outweighed by implementation cost or downside risk.

Sample size cannot be inferred from one fabricated path of observations. To plan a study, specify plausible data-generating conditions, a decision rule, stopping opportunities, and the required probability of reaching a useful decision. Evaluate those operating properties across possible datasets. A posterior probability can decrease when new observations arrive, and a first threshold crossing need not remain above the threshold. Keeping the rule and the assumptions explicit makes repeated monitoring interpretable.

Two cautions about thresholds specifically. A posterior probability crossing 0.95 is not the same event as a significant result, and the two rules have different error behavior under repeated looks — a Bayesian threshold is not automatically immune to the cost of peeking, which depends on the stopping rule and the prior. And the magnitude matters separately: a posterior probability of 0.99 that an effect is positive says nothing about whether the effect is large enough to act on. That question needs the posterior over the effect size, not a probability that it clears zero.

References

• SciPy documentation, scipy.stats.binom. docs.scipy.org
• SciPy documentation, scipy.stats.beta. docs.scipy.org