Sequential Settings

A sequential setting is any experimental or statistical framework in which data arrives over time and decisions can be made as it accumulates — stop or continue, accept or reject, shift traffic — rather than only at a predetermined end. It is the opposite of a fixed-horizon setting, where the sample size is set in advance and the data is analyzed once.

Its characteristics follow from that. Observations arrive one at a time or in small batches. Decisions can adapt: stopping early, changing allocation. In a frequentist framework the Type I error rate inflates if the accumulation is not handled properly. And specialized methods exist for exactly this reason.

The settings where it arises are the ones where waiting is expensive. Clinical trials recruit patients over time, analyze at interim checkpoints, and stop early if a treatment is clearly effective, harmful or futile. A/B tests see users arrive one by one while the people running them want to look. Online learning and bandit algorithms are sequential by construction, allocating more traffic to better-performing variants as evidence accumulates.

What the efficiency claim is worth

“Stop early, save time and cost” is the usual justification. It can be measured against the fixed-horizon design it replaces, at matched error rates.

import numpy as np
from scipy.stats import norm

p0, p1 = 0.10, 0.12
alpha, power = 0.05, 0.80
za, zb = norm.ppf(1 - alpha), norm.ppf(power)
n_fixed = int(np.ceil(((za * np.sqrt(p0 * (1 - p0))
                        + zb * np.sqrt(p1 * (1 - p1))) / (p1 - p0)) ** 2))
"""A fixed-horizon design for 10% against 12%, one-sided, 80% power."""

rng = np.random.default_rng(0)
runs, cap = 30_000, 6_000
A, Bnd = (1 - 0.20) / alpha, 0.20 / (1 - alpha)
la, lb = np.log(p1 / p0), np.log((1 - p1) / (1 - p0))

def sprt(p, runs, cap):
    x = (rng.random((runs, cap)) < p)
    llr = np.cumsum(np.where(x, la, lb), axis=1)
    up = llr >= np.log(A)
    dn = llr <= np.log(Bnd)
    hit = up | dn
    first = np.where(hit.any(axis=1), hit.argmax(axis=1), cap - 1)
    stopped_up = up[np.arange(runs), first]
    return first + 1, stopped_up, hit.any(axis=1)

print(f"  fixed-horizon design: {n_fixed:,} observations, one look at the end")
print(f"  SPRT thresholds A = {A:.1f}, B = {Bnd:.4f}")
print(f"{'truth':>14}{'reject H0':>12}{'mean n':>10}{'median n':>11}{'90th pct n':>13}"
      f"{'reached cap':>13}")
for name, p in (("p = 0.10 (H0)", p0), ("p = 0.12 (H1)", p1)):
    n, up, done = sprt(p, runs, cap)
    print(f"{name:>14}{up.mean():>12.4f}{n.mean():>10.1f}{np.median(n):>11.0f}"
          f"{np.quantile(n, 0.90):>13.0f}{1 - done.mean():>13.4f}")
print(f"  the mean SPRT run is shorter than {n_fixed:,}.")

pa, pb = 0.10, 0.20
print(f"  Two arms at {pa} and {pb}, {10_000:,} users, {1_000:,} repetitions:")
T, reps = 10_000, 1_000
res = {}
for label in ("even split", "UCB1"):
    g = np.random.default_rng(7)
    s_ = np.zeros((reps, 2)); n_ = np.zeros((reps, 2))
    idx = np.arange(reps)
    for tt in range(T):
        if label == "even split":
            arm = np.full(reps, tt & 1)
        elif tt < 2:
            arm = np.full(reps, tt)
        else:
            with np.errstate(divide="ignore", invalid="ignore"):
                score = s_ / n_ + np.sqrt(2 * np.log(tt) / n_)
            arm = np.argmax(score, axis=1)
        win = g.random(reps) < np.where(arm == 1, pb, pa)
        s_[idx, arm] += win
        n_[idx, arm] += 1
    est = s_[:, 1] / np.maximum(n_[:, 1], 1) - s_[:, 0] / np.maximum(n_[:, 0], 1)
    res[label] = (s_.sum(axis=1).mean(), n_[:, 1].mean() / T, est.mean(), est.std(ddof=1))
"""UCB1 is deterministic given the reward stream, and both policies here see
   the same stream, so the comparison is paired."""

print(f"{'policy':>14}{'conversions':>14}{'regret':>10}{'share on B':>13}"
      f"{'estimated lift':>17}{'sd of estimate':>17}")
for label, (rew, share, em, es) in res.items():
    print(f"{label:>14}{rew:>14.1f}{T * pb - rew:>10.1f}{share:>13.4f}"
          f"{em:>17.6f}{es:>17.6f}")
print(f"  the true lift is {pb - pa:.2f} and the best possible take is"
      f" {T * pb:,.0f} conversions.")
print(f"  The bandit gives up {T * pb - res['UCB1'][0]:.0f} of them against the even split's"
      f" {T * pb - res['even split'][0]:.0f},")
print(f"  and estimates the difference with {res['UCB1'][3] / res['even split'][3]:.1f}"
      " times the standard deviation, as allocation favors B.")
#   fixed-horizon design: 1,471 observations, one look at the end
#   SPRT thresholds A = 16.0, B = 0.2105
#          truth   reject H0    mean n   median n   90th pct n  reached cap
#  p = 0.10 (H0)      0.0470     687.2        489         1459       0.0000
#  p = 0.12 (H1)      0.8002     929.6        751         1813       0.0000
#   the mean SPRT run is shorter than 1,471.
#   Two arms at 0.1 and 0.2, 10,000 users, 1,000 repetitions:
#         policy   conversions    regret   share on B   estimated lift   sd of estimate
#     even split        1500.1     499.9       0.5000         0.100191         0.007231
#           UCB1        1913.7      86.3       0.9121         0.101403         0.011384
#   the true lift is 0.10 and the best possible take is 2,000 conversions.
#   The bandit gives up 86 of them against the even split's 500,
#   and estimates the difference with 1.6 times the standard deviation, as allocation favors B.

Two findings, and they are different findings

The first block prices the efficiency. A fixed design for 10% against 12% at one-sided 5% and 80% power needs 1,471 observations and looks once. The SPRT with thresholds set from the same \(\alpha\) and \(\beta\) uses approximate Wald boundaries, whose achieved errors need checking, and finishes in 687 observations on average under the null and 930 under the alternative — less than half and under two thirds. That is a real saving and it is the argument.

The number to notice alongside it is the 90th percentile: 1,459 and 1,813 observations. The saving is an average, and one run in ten is about as long as the fixed design or longer. A sequential design trades a known duration for a shorter expected one, which is a good trade when the cost is per observation and a poor one when the cost is a deadline.

The second block separates two things the phrase covers. A bandit and a test are both sequential and they optimize different quantities. Over 10,000 users on arms at 0.10 and 0.20, an even split gives up 500 conversions against always playing the better arm; UCB1 gives up 86. But the bandit sends 91% of its traffic to one arm, so it estimates the difference 1.6 times less precisely — it has stopped collecting the data that would measure what it is exploiting. A test buys a number you can defend; a bandit buys revenue during the experiment. Choosing a sequential method is choosing which of those you are paying for, and a bandit run to decide a question is a common way to end up with neither.

The methods, grouped by what they protect

Frequentist approaches control an error rate over the whole procedure. The SPRT compares the likelihood ratio against two thresholds after each observation and stops when either is crossed. Group sequential tests — O’Brien–Fleming, Pocock — analyze at pre-planned checkpoints and spend the significance level across them, which is what keeps the overall rate at its nominal value.

Bayesian approaches update a posterior as data arrives and stop when it crosses a threshold, such as a 95% posterior probability of improvement. Exact posterior updates are coherent under the specified model, fixed prior and ignorable stopping; approximations need accuracy checks; what the threshold does not give is a calibrated long-run error rate, since that belongs to the whole rule and must be calculated, bounded or simulated if it is a requirement.

Adaptive and online methods optimize the decisions themselves rather than the inference: multi-armed bandits such as \(\varepsilon\)-greedy and Thompson sampling, and reinforcement learning more broadly. They are the right tool when the traffic during the experiment is the thing being optimized, and can also support inference with allocation-aware estimators and confidence sequences. Naive fixed-sample intervals are not automatically valid after adaptive allocation.

Why the setting matters

Efficiency, as measured above. Ethics, because a trial should not continue once a treatment is clearly harmful or clearly better. Business responsiveness, because an experiment that runs to a fixed horizon spends traffic on a variant already known to be worse. And necessity, because many learning algorithms only ever see data as a stream.

What every method in the list has in common is that the stopping rule is part of the procedure. Analyzing sequentially with a tool designed for one look is the failure mode, and the specialized methods exist to make the rule explicit rather than to remove the need for one.

References

• SciPy documentation, scipy.stats.norm. docs.scipy.org