1. Definition

  • A sequential setting is any experimental or statistical framework where data arrives over time and decisions (e.g., stop/continue, accept/reject, allocate traffic) can be made as data accumulates, rather than only at a fixed end point.
  • Opposite of a fixed-horizon setting, where you predefine a sample size and only analyze at the end.

In short:
Sequential setting = analyze or decide while data is streaming in, not just at the end.

2. Characteristics of Sequential Settings

  • Data comes sequentially (one observation at a time, or in small batches).
  • Decisions can be adaptive (e.g., stopping early, changing allocations).
  • Risk of Type I error inflation if not handled properly in frequentist frameworks.
  • Requires specialized statistical methods (SPRT, group sequential, Bayesian, bandits).

3. Examples

(a) Clinical Trials

  • Patients recruited over time.
  • Data is analyzed at interim checkpoints.
  • Stopping rules: stop early if drug is effective/harmful/futile.

(b) A/B Testing

  • Users arrive to a website one by one.
  • Marketers want to peek at results mid-experiment.
  • Sequential methods (Bayesian, group sequential, bandits) allow safe interim looks.

(c) Machine Learning (Online Learning / Bandits)

  • Algorithms learn sequentially as data streams in.
  • Example: multi-armed bandit → allocate more traffic to better-performing variants over time.

4. Statistical Approaches in Sequential Settings

  1. Frequentist Approaches
    • SPRT (Sequential Probability Ratio Test): Check likelihood ratios continuously, stop if thresholds are crossed.
    • Group Sequential Tests (O’Brien-Fleming, Pocock): Analyze at pre-planned checkpoints with α-spending.
  2. Bayesian Approaches
    • Natural for sequential settings.
    • Posterior probability can be updated continuously as data arrives.
    • Stopping rules: stop when posterior probability crosses a threshold (e.g., >95%).
  3. Adaptive/Online Learning
    • Multi-armed bandits (ε-greedy, Thompson sampling).
    • Reinforcement learning.
    • Designed for sequential decision-making.

5. Why Sequential Settings Matter

  • Efficiency: Stop experiments early → save time and cost.
  • Ethics: In clinical trials, don’t continue if a treatment is clearly harmful.
  • Business: In online A/B testing, adapt quickly to user behavior.
  • Machine Learning: Many algorithms assume data arrives sequentially.

6. Key Takeaways

  • Sequential settings = data and decisions happen over time, not just at a fixed horizon.
  • Requires special methods to avoid bias and false positives.
  • Widely used in clinical trials, A/B testing, and online learning.

In short:
A sequential setting is when data arrives over time and you can analyze or adapt decisions as it comes in. It contrasts with fixed-horizon settings and requires specialized methods (SPRT, Bayesian updating, bandits, group sequential tests) to maintain statistical validity.