1. Definition

  • A stopping rule is the predefined condition under which an experiment or test is stopped.
  • It tells you when to stop collecting data and make a decision.
  • Without clear stopping rules, you risk inflating Type I error (false positives) by “peeking” until results look significant.

2. Types of Stopping Rules

(a) Fixed-Horizon Stopping Rule (Traditional A/B Test)

  • Decide sample size beforehand (e.g., 10,000 users per group).
  • Stop once sample is reached.
  • One analysis at the end.
  • Simple, protects α, but inflexible.

(b) Group Sequential Stopping Rules

  • Pre-plan interim looks at data (e.g., every 25% of sample).
  • Use α-spending rules (like O’Brien–Fleming, Pocock).
  • Can stop early if results are overwhelming (efficacy) or hopeless (futility).
  • More efficient, widely used in clinical trials.

(c) Sequential Probability Ratio Test (SPRT)

  • Evaluate likelihood ratio after each data point.
  • If ratio exceeds thresholds → stop for H₁ or H₀.
  • Otherwise → keep sampling.
  • Often requires fewer samples than fixed designs.

(d) Bayesian Stopping Rules

  • Stop when posterior probability of H₁ (or H₀) exceeds a threshold.
  • Example: Stop when $P(H_1 \mid \text{data}) > 0.95$.
  • More intuitive (gives probability statements), no α-spending needed.

(e) Ethical / Practical Stopping Rules

  • In clinical trials:
    • Stop if treatment shows harm (safety).
    • Stop if treatment shows clear benefit (unethical to withhold).
    • Stop if treatment shows futility (unlikely to ever show benefit).

3. Examples

A/B Test Example

  • Fixed rule: Run until 50,000 visitors per group.
  • Sequential rule: Check every 5,000 visitors. Stop early if p < 0.001 (OBF rule).
  • Bayesian rule: Stop when posterior probability > 0.95 that B > A.

Clinical Trial Example

  • Planned for 1,000 patients.
  • Interim checks after 250, 500, 750.
  • Stopping rules:
    • If new drug reduces mortality significantly early → stop.
    • If harm observed → stop immediately.
    • If no chance of benefit → stop for futility.

4. Why Stopping Rules Matter

  • Prevent p-hacking (stopping as soon as results look “good”).
  • Ensure correct Type I error control.
  • Save time, cost, and resources (stop early if results are clear).
  • Protect patients/participants (clinical trials).

5. Key Takeaways

  • Stopping rules = when and how you end an experiment.
  • They can be fixed-horizon (traditional) or sequential/adaptive (group sequential, SPRT, Bayesian).
  • Good stopping rules protect against false positives while improving efficiency and ethics.

In short:
Stopping rules are pre-specified conditions for when to stop an experiment. Traditional rules use a fixed sample size, while modern sequential and Bayesian rules allow interim looks or continuous monitoring without inflating false positives.