Beta Distribution

The Beta distribution is a continuous distribution on \([0, 1]\) with two shape parameters:

\[ f(p \mid \alpha, \beta) = \frac{1}{B(\alpha, \beta)}\,p^{\alpha - 1}(1 - p)^{\beta - 1} \]

Because its support is exactly the range a probability can occupy, it is the default distribution for a proportion — a conversion rate, a click-through rate, a defect rate.

Two parameters, many shapes

import numpy as np
from scipy.stats import beta

print(f"{'parameters':>18}{'mean':>9}{'mode':>21}{'sd':>9}{'shape':>30}")
for a, b, shape in ((1, 1, "flat - Uniform(0,1)"), (0.5, 0.5, "U-shaped, mass at the ends"),
                    (5, 2, "skewed toward 1"), (2, 5, "skewed toward 0"),
                    (20, 20, "tight around 0.5"), (2, 1, "increasing, no interior mode")):
    d = beta(a, b)
    mode = (a - 1) / (a + b - 2) if a > 1 and b > 1 else float("nan")
    shown = f"{mode:.4f}" if np.isfinite(mode) else "none (a or b <= 1)"
    print(f"{f'Beta({a}, {b})':>18}{d.mean():>9.4f}{shown:>21}{d.std():>9.4f}{shape:>30}")
"""Mean is a/(a+b); the mode is (a-1)/(a+b-2) only when there is one."""

print(f"{'prior':>16}{'a + b':>9}{'prior mean':>13}{'after 7 of 10':>16}{'after 70 of 100':>18}")
for a, b in ((1, 1), (2, 2), (20, 20), (1, 9)):
    post = lambda k, n: (a + k) / (a + b + n)
    print(f"{f'Beta({a}, {b})':>16}{a + b:>9}{a / (a + b):>13.4f}"
          f"{post(7, 10):>16.4f}{post(70, 100):>18.4f}")

for a, b in ((1, 1), (5, 5), (50, 50), (500, 500)):
    d = beta(a, b)
    print(f"    Beta({a:>3}, {b:>3}) mean {d.mean():.4f}  sd {d.std():.6f}"
          f"  95% interval [{d.ppf(0.025):.4f}, {d.ppf(0.975):.4f}]")

for a, b in ((0.8, 0.2), (4, 1), (400, 100)):
    d = beta(a, b)
    print(f"    Beta({a:>5}, {b:>5}) mean {d.mean():.4f}  "
          f"P(0.7 < p < 0.9) = {d.cdf(0.9) - d.cdf(0.7):.4f}")
#         parameters     mean                 mode       sd                         shape
#         Beta(1, 1)   0.5000   none (a or b <= 1)   0.2887           flat - Uniform(0,1)
#     Beta(0.5, 0.5)   0.5000   none (a or b <= 1)   0.3536    U-shaped, mass at the ends
#         Beta(5, 2)   0.7143               0.8000   0.1597               skewed toward 1
#         Beta(2, 5)   0.2857               0.2000   0.1597               skewed toward 0
#       Beta(20, 20)   0.5000               0.5000   0.0781              tight around 0.5
#         Beta(2, 1)   0.6667   none (a or b <= 1)   0.2357  increasing, no interior mode
#            prior    a + b   prior mean   after 7 of 10   after 70 of 100
#       Beta(1, 1)        2       0.5000          0.6667            0.6961
#       Beta(2, 2)        4       0.5000          0.6429            0.6923
#     Beta(20, 20)       40       0.5000          0.5400            0.6429
#       Beta(1, 9)       10       0.1000          0.4000            0.6455
#     Beta(  1,   1) mean 0.5000  sd 0.288675  95% interval [0.0250, 0.9750]
#     Beta(  5,   5) mean 0.5000  sd 0.150756  95% interval [0.2120, 0.7880]
#     Beta( 50,  50) mean 0.5000  sd 0.049752  95% interval [0.4027, 0.5973]
#     Beta(500, 500) mean 0.5000  sd 0.015803  95% interval [0.4690, 0.5310]
#     Beta(  0.8,   0.2) mean 0.8000  P(0.7 < p < 0.9) = 0.1512
#     Beta(    4,     1) mean 0.8000  P(0.7 < p < 0.9) = 0.4160
#     Beta(  400,   100) mean 0.8000  P(0.7 < p < 0.9) = 1.0000

The first table is the gallery. Beta(1, 1) is exactly Uniform(0, 1). Beta(0.5, 0.5) is U-shaped, putting mass at both ends rather than the middle. Beta(5, 2) leans toward 1 and Beta(2, 5) toward 0, mirror images with identical spread. Beta(20, 20) is tight around 0.5.

Note the mode column. The familiar formula \((\alpha - 1)/(\alpha + \beta - 2)\) requires both parameters above 1, and three rows return nothing — not a computational failure but a correct answer, since those densities have no interior peak. Beta(2, 1) increases monotonically to its right edge; Beta(0.5, 0.5) has two peaks at the boundaries.

The mean \(\alpha/(\alpha + \beta)\) always exists and is the safer summary. A distribution described only by its mode is underdescribed, and one described only by its mean can hide anything — the last block shows Beta(0.8, 0.2), Beta(4, 1) and Beta(400, 100) all sitting at a mean of 0.8000 while \(P(0.7 < p < 0.9)\) runs 0.1512, 0.4160, 1.0000.

Reading the parameters as pseudo-counts

The second block is why this distribution and the binomial belong together. A Beta\((\alpha, \beta)\) prior updated by \(k\) successes in \(n\) trials gives a Beta\((\alpha + k, \beta + n - k)\) posterior — the update is addition, and nothing else has to be computed.

That is conjugacy, and the posterior mean makes \(\alpha + \beta\) readable as a pseudo-count weight on the prior mean. This weight is an interpretation of that update, not a unique information measure for every purpose. Beta(20, 20) behaves like forty prior tosses: after 7 of 10 it has moved only to 0.5400, while a Beta(1, 1) prior worth two observations has moved to 0.6667. The practical reading is that choosing a Beta prior is choosing how many imaginary observations to put on the scale.

The third block shows the spread behaving as any estimate should: at large concentration, a tenfold increase in weight cuts the standard deviation by roughly \(\sqrt{10}\), The displayed sequence starts with a fivefold increase from Beta(1, 1) to Beta(5, 5), then two tenfold increases; its standard deviations run from 0.288675 to 0.015803. The interval narrows accordingly.

Where it fits and where it does not

Three priors recur and each encodes a different claim. Beta(1, 1) asserts every rate equally plausible. Beta(0.5, 0.5) — the Jeffreys prior for a proportion — is invariant under reparameterization, which the uniform prior is not, and it leans toward the extremes. Beta\((a, b)\) fitted to a historical rate encodes real knowledge and has to be defended as such.

The limits are worth naming. The Beta describes a single proportion; a Dirichlet describes a vector of category probabilities summing to one, not arbitrary correlated proportions. It is continuous, so it models the underlying rate rather than the observed count — the count is binomial. And conjugacy is a property of the Beta–binomial pairing rather than a general fact: a different likelihood may lose that closed form. Numerical integration, variational approximation or sampling are then possible computational approaches.

One convenience deserves a caution. The closed form makes it tempting to reach for a Beta prior because the arithmetic is easy rather than because the belief fits, and an easy prior still carries \(\alpha + \beta\) observations of weight into the result.

References

• SciPy documentation, scipy.stats.beta. docs.scipy.org