O’Brien–Fleming (OBF) Method

The O'Brien–Fleming boundary is the group sequential design that spends almost none of the error budget early and nearly all of it at the end. Its critical value falls as information accumulates, which is the opposite shape to Pocock's flat line.

The construction is one constant divided by the square root of the information fraction: \(c/\sqrt{k/K}\). That single expression produces the whole shape.

The shape, and its numbers

import numpy as np
from scipy.stats import norm

rng = np.random.default_rng(0)
N = 400_000
OBF = {2: [2.794, 1.977], 3: [3.471, 2.454, 2.004],
       4: [4.049, 2.863, 2.338, 2.024], 5: [4.562, 3.226, 2.634, 2.281, 2.040]}

def z_at_looks(K, drift=0.0):
    steps = rng.normal(size=(N, K)) / np.sqrt(K)
    t = np.arange(1, K + 1) / K
    return (np.cumsum(steps, axis=1) + drift * 2.8016 * t) / np.sqrt(t)

print(f"{'looks':>7}{'boundary, first to last':>46}{'measured type I':>18}")
for K, b in OBF.items():
    rate = float((np.abs(z_at_looks(K)) >= np.asarray(b)).any(axis=1).mean())
    print(f"{K:>7}{str(b):>46}{rate:>18.4f}")
"""The boundary falls as information accumulates: strict when little is known,
   close to the ordinary threshold at the end."""

print(f"\n{'look':>6}{'fraction of data':>19}{'z threshold':>14}{'nominal p':>13}")
for k, b in enumerate(OBF[4], start=1):
    print(f"{k:>6}{k / 4:>19.2f}{b:>14.3f}{2 * (1 - norm.cdf(b)):>13.6f}")
print(f"  final nominal two-sided p threshold: {2 * (1 - norm.cdf(2.024)):.4f}")

print(f"\n{'design':>28}{'power':>9}{'average looks':>15}{'stops at look 1':>17}")
for name, bounds in (("fixed horizon", [np.inf, np.inf, np.inf, 1.960]),
                     ("Pocock, 4 looks", [2.361] * 4),
                     ("O'Brien-Fleming, 4 looks", OBF[4])):
    z = z_at_looks(4, drift=1.0)
    crossed = np.abs(z) >= np.asarray(bounds)
    first = np.where(crossed.any(axis=1), crossed.argmax(axis=1) + 1, 4)
    print(f"{name:>28}{crossed.any(axis=1).mean():>9.4f}{first.mean():>15.2f}"
          f"{float((first == 1).mean()):>17.4f}")
#   looks                       boundary, first to last   measured type I
#       2                                [2.794, 1.977]            0.0501
#       3                         [3.471, 2.454, 2.004]            0.0496
#       4                  [4.049, 2.863, 2.338, 2.024]            0.0500
#       5            [4.562, 3.226, 2.634, 2.281, 2.04]            0.0501
#
#   look   fraction of data   z threshold    nominal p
#      1               0.25         4.049     0.000051
#      2               0.50         2.863     0.004197
#      3               0.75         2.338     0.019387
#      4               1.00         2.024     0.042970
#   final nominal two-sided p threshold: 0.0430
#
#                       design    power  average looks  stops at look 1
#                fixed horizon   0.7986           4.00           0.0000
#              Pocock, 4 looks   0.7188           2.87           0.1684
#     O'Brien-Fleming, 4 looks   0.7905           3.27           0.0039

The second table is the one to read carefully, because the values circulate in a corrupted form. For a four-look design at two-sided 0.05, the boundary is 4.049, 2.863, 2.338, 2.024, with nominal p-values 0.000051, 0.004197, 0.019387 and 0.042970. The first table confirms these give 0.0500 overall.

At 25% information, z = 4.049 corresponds to a two-sided p-value of about one in twenty thousand. A null process can cross this boundary, but rarely. The first-look stopping rate of 0.39% below belongs to the simulated target effect. Statistical crossing and an ethical decision to stop are different judgments.

The other end is the reason it is the usual default. The final look uses a nominal two-sided p threshold of 0.042970, close to the ordinary 0.05 threshold. This is not the incremental alpha spent at that look: overall spending depends on crossing probabilities across the correlated analyses. A study that runs to completion is barely penalized for having had the option to stop, which is what the power column in the last table shows: 0.7905 against a fixed design's 0.7986.

Reading the constants correctly

The boundary depends on how many looks were planned, and the values for different \(K\) look similar enough to be mixed up. The first table gives them side by side: the first-look threshold is 2.794 for two looks, 3.471 for three, 4.049 for four, 4.562 for five. A widely copied “four-look” table that opens at 3.471 is the three-look boundary wearing the wrong label.

The second analysis in the equally spaced three-look design occurs at two-thirds information, not 50%. Its nominal p-value near 0.014 therefore cannot be compared as though it were the four-look design at 50% information, where the nominal threshold is about 0.0042. Report both the number of looks and their information fractions.

In practice most implementations do not use a fixed table at all. The Lan–DeMets spending-function formulation computes the boundary from the information accrued at each analysis, which is what lets the looks happen when they happen rather than exactly at the planned fractions.

Where it belongs

The last table sets it against the alternatives at the effect the study was sized for. O'Brien–Fleming stops at the first look 0.39% of the time against Pocock's 16.84%, uses 3.27 of four blocks against Pocock's 2.87, and gives up one point of power against Pocock's eight.

The low early alpha expenditure can be useful when preserving the final analysis matters more than frequent early stopping. An efficacy boundary alone does not establish that continuing a trial is unethical: that judgment also depends on benefit, harm, uncertainty and the monitoring protocol.

In business experimentation the same judgment often points the other way. If the value of stopping early is a month of engineering time rather than patient safety, a boundary that almost never stops early is providing an option nobody exercises. Choose the shape from what an early stop is worth to you, and keep the honest accounting in view: neither design is free, and both control the same 0.05.

References

• SciPy documentation, scipy.stats.norm. docs.scipy.org
• NumPy documentation, numpy.random.Generator.normal. numpy.org