Bootstrap Confidence Intervals (CIs)

1. What They Are

• A bootstrap confidence interval is a way to estimate the uncertainty of a statistic (mean, median, regression coefficient, etc.) using resampling, rather than relying on strict parametric formulas.
• Idea: If you don’t know the population distribution, approximate it by resampling the observed data many times.

It’s very useful when:

• Sample sizes are small.
• Data are non-normal.
• No simple analytical formula for standard error exists.

---

2. Bootstrap Procedure

1. Take your original dataset of size $n$.
2. Resample with replacement $B$ times (e.g., $B=1000$) to create bootstrap samples.
3. For each sample, compute the statistic of interest (e.g., mean).
4. Collect the distribution of these bootstrap statistics.
5. Use that distribution to form confidence intervals.

---

3. Types of Bootstrap CIs

There are several ways to build CIs from bootstrap samples:

(a) Percentile Method

• Take the $\alpha/2$ and $1-\alpha/2$ quantiles from the bootstrap distribution.
• Example: 95% CI = [2.5th percentile, 97.5th percentile].

(b) Basic (Reverse Percentile) Method

• Uses bias correction by reflecting the percentile interval around the observed statistic.

(c) BCa (Bias-Corrected and Accelerated) Method

• Adjusts for both bias and skewness in the bootstrap distribution.
• Often recommended as the most reliable.

---

4. Example

Say we have 10 data points:

$X = [5, 7, 9, 10, 12, 8, 6, 7, 9, 11]$

• Statistic of interest: mean = 8.4
• Bootstrap: Generate 1000 resamples (size 10 each), compute the mean each time.
• Bootstrap means distribution might look roughly normal, centered near 8.4.

From that:

• 2.5th percentile = 7.2
• 97.5th percentile = 9.6

→ 95% bootstrap CI for the mean = [7.2, 9.6]

---

5. Advantages

• Doesn’t require normality assumptions.
• Works with complex statistics (median, regression coefficients, AUC, etc.).
• Easy to implement with modern computing.

6. Limitations

• Computationally expensive (requires thousands of resamples).
• Can still be biased if sample is very small or not representative.
• Different methods (percentile, BCa, etc.) may give slightly different intervals.

---

Summary:
Bootstrap CIs = resample your data many times, compute the statistic each time, and take quantiles of the distribution to form confidence intervals.
They’re flexible, non-parametric, and especially useful when standard formulas don’t apply.