Cumulative Distribution Function (CDF)

The cumulative distribution function gives, for every \(x\), the probability that the variable is at most \(x\): \(F(x) = P(X \le x)\). It is non-decreasing, runs from 0 to 1, and exists for every random variable — discrete, continuous, or a mixture of both.

That universality is why it, rather than a density, is the definition a distribution is built on.

One function, every question

import numpy as np
from scipy.stats import norm, poisson

F = norm(0, 1).cdf
print(f"  the CDF answers one question at every point: P(X <= x)")
print(f"{'x':>7}{'F(x)':>12}{'P(X > x)':>12}{'F(x) - F(-x)':>16}")
for x in (-2, -1, 0, 1, 2, 3):
    print(f"{x:>7}{F(x):>12.6f}{1 - F(x):>12.6f}{F(x) - F(-x):>16.6f}")
"""Every interval probability is a difference of two CDF values, so the CDF
   carries the whole distribution - nothing else is needed."""

print(f"\n  it works for discrete variables too, where a density does not exist:")
print(f"{'k':>5}{'F(k) for Poisson(2)':>23}{'jump at k = P(X = k)':>24}")
prev = 0.0
for k in range(0, 5):
    now = poisson.cdf(k, 2.0)
    print(f"{k:>5}{now:>23.6f}{now - prev:>24.6f}")
    prev = now
print("  a discrete CDF is a staircase, and the height of each step is the mass.")
print("  a continuous CDF has no steps, which is P(X = x) = 0 stated differently")

rng = np.random.default_rng(0)
u = rng.random(200_000)
draws = norm.ppf(u)
print(f"\n  because F runs from 0 to 1, its inverse turns uniform noise into any")
print(f"  distribution you like - this is how samplers are built:")
print(f"{'target quantile':>17}{'from the inverse CDF':>23}{'measured in the draws':>24}")
for q in (0.10, 0.50, 0.90, 0.99):
    print(f"{q:>17}{norm.ppf(q):>23.4f}{np.quantile(draws, q):>24.4f}")
print(f"\n  and the same inverse is what a percentile is: the 95th percentile of")
print(f"  a standard normal is {norm.ppf(0.95):.4f}, which is F inverted at 0.95")
#   the CDF answers one question at every point: P(X <= x)
#       x        F(x)    P(X > x)    F(x) - F(-x)
#      -2    0.022750    0.977250       -0.954500
#      -1    0.158655    0.841345       -0.682689
#       0    0.500000    0.500000        0.000000
#       1    0.841345    0.158655        0.682689
#       2    0.977250    0.022750        0.954500
#       3    0.998650    0.001350        0.997300
#
#   it works for discrete variables too, where a density does not exist:
#     k    F(k) for Poisson(2)    jump at k = P(X = k)
#     0               0.135335                0.135335
#     1               0.406006                0.270671
#     2               0.676676                0.270671
#     3               0.857123                0.180447
#     4               0.947347                0.090224
#   a discrete CDF is a staircase, and the height of each step is the mass.
#   a continuous CDF has no steps, which is P(X = x) = 0 stated differently
#
#   because F runs from 0 to 1, its inverse turns uniform noise into any
#   distribution you like - this is how samplers are built:
#   target quantile   from the inverse CDF   measured in the draws
#               0.1                -1.2816                 -1.2853
#               0.5                 0.0000                 -0.0045
#               0.9                 1.2816                  1.2737
#              0.99                 2.3263                  2.3315
#
#   and the same inverse is what a percentile is: the 95th percentile of
#   a standard normal is 1.6449, which is F inverted at 0.95

The first table shows the arithmetic. \(P(X > x)\) is \(1 - F(x)\), and any interval probability is a difference of two values of \(F\) — the last column reads \(F(x) - F(-x)\), which at \(x = 1\) gives the familiar 0.682689 and at \(x = 2\) gives 0.954500. For negative \(x\) the same expression is negative, which is a reminder that it is a difference and not a rule: the symmetric interval is read at positive \(x\).

Nothing else is needed. Interval probabilities are obtained from differences of \(F\) values, with the endpoint convention made explicit; for example, \(P(adistribution.

The second table is the reason this is the general definition. For a Poisson the CDF is a staircase, and the height of each step is exactly the probability mass at that value — 0.135335 at 0, 0.270671 at 1. A continuous CDF has no steps, which is \(P(X = x) = 0\) stated in a different vocabulary. The same object describes both, and where the CDF is differentiable its slope is the density.

The inverse is where the work happens

Because \(F\) is non-decreasing and runs from 0 to 1, it can be inverted. The third block feeds uniform random numbers through \(F^{-1}\) for a normal and recovers the target quantiles — 0.10 at −1.2816 against a measured −1.2853, 0.99 at 2.3263 against 2.3315.

That is inverse transform sampling, and it is how a great many random number generators are built: get uniform noise, push it through the inverse CDF of whatever you want. It works for any distribution whose inverse CDF you can evaluate, which is one more practical reason the CDF is the object software exposes.

The same inverse is what a quantile is. The 95th percentile of a standard normal is \(F^{-1}(0.95) = 1.6449\); a median is \(F^{-1}(0.5)\); a confidence interval's multiplier is a pair of these. Percentiles are not a separate concept bolted onto distributions — they are the CDF read backwards.

Two details worth knowing

For a discrete variable the inverse needs a convention, because \(F\) skips values: there is no \(x\) with \(F(x) = 0.5\) if the staircase jumps over it. The standard definition takes the smallest \(x\) with \(F(x) \ge q\), which is why quantiles of count data land on integers and why different software can disagree about a sample percentile by one position.

And the empirical CDF — the staircase built from a sample, stepping by \(1/n\) at each observation — is the sample equivalent, used directly by the Kolmogorov–Smirnov test and by every quantile a dataset reports. Under independent, identically distributed sampling, it converges to the true CDF uniformly as the sample grows, which is the formal statement behind the practice of reading percentiles off data and trusting them.

References

• SciPy documentation, scipy.stats.norm (cdf, ppf). docs.scipy.org
• NumPy documentation, numpy.quantile. numpy.org