Wilson Score Interval

The Wilson score interval is a confidence interval for a proportion built by inverting the hypothesis test rather than by adding a symmetric margin to the estimate. The NIST handbook describes it that way — the method, “introduced by Wilson (1927), recommended by Brown, Cai and DasGupta (2001) and Agresti and Coull (1998)), is based on inverting the hypothesis test” — and gives two reasons to prefer it.

What is wrong with the usual interval

The interval most people learn is \(\hat{p} \pm z_{1-\alpha/2}\sqrt{\hat{p}(1-\hat{p})/n}\), and it has a defect the handbook states bluntly. The Wilson interval’s “advantage is that the lower limit cannot be negative. That is not true for the confidence expression most frequently used” — followed by the verdict: “a confidence limit approach that produces a lower limit which is an impossible value for the parameter for which the interval is constructed is an inferior approach.“

That is not a corner case in product work. Three conversions in 300 independent sessions give a 1% observed rate; the usual 95% Wald interval extends below zero. Intersecting that interval with \([0,1]\) removes impossible values but does not repair its coverage. For any true proportion in \([0,1]\), the original and clipped intervals contain it on exactly the same samples. A method such as Wilson addresses the interval construction itself.

The second advantage is about robustness across situations: “its worth does not strongly depend upon the value of \(n\) and/or \(p\), and indeed was recommended by Agresti and Coull for virtually all combinations of \(n\) and \(p\).” So it is not a special-case tool for small samples — it is a reasonable default that happens to also behave at the edges.

To invert the score test, try a null proportion p₀ and compute z(p₀) = (p̂ − p₀)/√[p₀(1 − p₀)/n]. Unlike the Wald standard error, this denominator uses the candidate null rate. Keep the p₀ values that the two-sided test does not reject, |z(p₀)| ≤ 1.96 at 95%; solving the boundary equations gives the Wilson interval. For 3 successes in 300 trials, those bounds are approximately 0.00341 and 0.02898, or 0.34% to 2.90%, whereas the Wald lower bound is about −0.13%. The inversion uses the normal score approximation, so bounds inside [0,1] do not guarantee exact nominal coverage at every binomial parameter.

How it differs in shape

Two structural differences follow from being derived by inversion rather than by addition.

• It is not centered on \(\hat{p}\). The interval is pulled toward 0.5, and the pull is larger when \(n\) is small. That asymmetry is the correction, not a flaw.
• It stays inside [0, 1]. Which means a reported bound is always a possible value of the thing being estimated.

A one-sided version needs no new formula: “a one-sided confidence interval can also be constructed simply by replacing each \(z_{\alpha/2}\) by \(z_\alpha\).” Useful when only one direction matters — an upper bound on a defect rate, a lower bound on a success rate. See confidence intervals for what the coverage statement means and confidence level for the number you choose.

When to use which

Three choices are available for a proportion, and the handbook’s own framing points at when the approximations stop being good enough: “if the number of failures is very small or if the sample size \(N\) is very small, symmetrical confidence limits that are approximated using the normal distribution may not be accurate enough for some applications.”

One caution that applies to all three: they describe sampling variability for a binomial count, so they assume independent trials. Sessions from the same user, or repeated visits, break that — and then the unit of analysis is the problem rather than the interval formula. See sample size and standard error. Where you need an interval for something that is not a simple proportion, resampling is the general answer — see bootstrap confidence intervals.

References

• NIST/SEMATECH e-Handbook of Statistical Methods, §7.2.4.1 “Confidence intervals”