Pocock Method

The Pocock design is a group sequential boundary that uses the same critical value at every look, including the last. It is the simplest shape a spending rule can take: one number, applied \(K\) times.

That number is not 1.96, and it is not a matter of preference. It is whatever value makes the probability of crossing at any of the \(K\) looks come out to 0.05.

One constant per design, not one constant

import numpy as np
from scipy.stats import norm

rng = np.random.default_rng(0)
N = 400_000
CONSTANT = {2: 2.178, 3: 2.289, 4: 2.361, 5: 2.413, 10: 2.555}

def z_at_looks(K, drift=0.0):
    steps = rng.normal(size=(N, K)) / np.sqrt(K)
    t = np.arange(1, K + 1) / K
    return (np.cumsum(steps, axis=1) + drift * 2.8016 * t) / np.sqrt(t)

print(f"{'looks':>7}{'Pocock z':>11}{'nominal p at every look':>26}{'measured type I':>18}")
for K, c in CONSTANT.items():
    rate = float((np.abs(z_at_looks(K)) >= c).any(axis=1).mean())
    print(f"{K:>7}{c:>11.3f}{2 * (1 - norm.cdf(c)):>26.5f}{rate:>18.4f}")
"""One constant, used at every look. The constant rises with the number of
   looks because the same 0.05 has to cover more chances to cross."""

for K, c in ((4, 2.361), (5, 2.413)):
    wrong = float((np.abs(z_at_looks(4)) >= c).any(axis=1).mean())
    print(f"    using the K={K} constant {c} in a 4-look design -> type I {wrong:.4f}")

print(f"\n{'design':>26}{'power':>9}{'average looks':>15}")
for name, bounds in (("fixed horizon", [np.inf, np.inf, np.inf, 1.960]),
                     ("Pocock, 4 looks", [2.361] * 4)):
    z = z_at_looks(4, drift=1.0)
    crossed = np.abs(z) >= np.asarray(bounds)
    first = np.where(crossed.any(axis=1), crossed.argmax(axis=1) + 1, 4)
    print(f"{name:>26}{crossed.any(axis=1).mean():>9.4f}{first.mean():>15.2f}")
print(f"  the final look is judged at p = {2 * (1 - norm.cdf(2.361)):.5f}, not 0.05:")
#   looks   Pocock z   nominal p at every look   measured type I
#       2      2.178                   0.02941            0.0501
#       3      2.289                   0.02208            0.0498
#       4      2.361                   0.01823            0.0499
#       5      2.413                   0.01582            0.0502
#      10      2.555                   0.01062            0.0497
#     using the K=4 constant 2.361 in a 4-look design -> type I 0.0499
#     using the K=5 constant 2.413 in a 4-look design -> type I 0.0442
#
#                     design    power  average looks
#              fixed horizon   0.8009           4.00
#            Pocock, 4 looks   0.7195           2.87
#   the final look is judged at p = 0.01823, not 0.05:

The first table is the whole method. Two looks need 2.178, four need 2.361, ten need 2.555 — and each returns a measured false positive rate of 0.05. The constant rises because the same budget has to cover more opportunities to cross.

The nominal p-value column is the number practitioners actually use: with four looks, each analysis is judged at 0.01823 rather than 0.05. Note that this applies to the final look too, which is the property that most surprises people and the one worth stating twice — a Pocock design judges its last analysis at 0.018. A result with p = 0.03 at the end of the study would have been significant in a fixed design and is not significant here.

The middle block makes the dependence explicit. Using the five-look constant 2.413 in a four-look design gives a false positive rate of 0.0442 instead of 0.0499. That error is in the safe direction, which is why it survives in circulated tables; it is still an error, because it leaves part of the 0.05 unspent and buys nothing for it. The constant belongs to the design, not to the method.

What it is good at

The last block shows the trade. Against a fixed design reaching power 0.8009 with all four blocks of data, Pocock reaches 0.7195 using 2.87 looks on average — a little under three quarters of the planned sample.

That is the case for it. A flat boundary is genuinely reachable at the first look, so a real effect of moderate size gets caught early and the study ends. Compared with O'Brien–Fleming, which almost never stops at the first analysis, Pocock is the design that actually delivers early stopping rather than the option of it.

A constant critical value is simple to operate once the information times and total two-sided alpha have been specified. Its calibration depends on those times as well as the number of looks. If information fractions change, recompute the boundaries or use an appropriate prespecified alpha-spending design; do not apply an equally spaced table at arbitrary times.

What it costs

The eight points of power in that table are the cost, and they land in a specific place: on studies that run to completion. If the effect is smaller than planned — which is the common case — the study goes the full distance and is then judged at 0.018 instead of 0.05. Pocock is the design that most often ends with a result that would have been significant under a fixed horizon and is not.

The standard remedy is to raise the planned maximum sample size so the overall power, including early rejection, matches what a fixed design would have had. That restores the power and means the saving exists only in expectation: a Pocock study that does not stop early uses more data than the fixed design it replaced.

The choice between the two shapes therefore comes down to a forecast. If you expect a clear effect and value stopping early, the flat boundary pays. If you expect a marginal effect and need the final analysis to be as powerful as possible, the declining boundary does. Neither is more correct; both control the same 0.05.

References

• SciPy documentation, scipy.stats.norm. docs.scipy.org
• NumPy documentation, numpy.random.Generator.normal. numpy.org
• gsDesign, Spending Function Overview. Source