1. Definition
- A Frequentist approach to statistics defines probability as the long-run frequency of events in repeated trials.
- Parameters (like population mean, conversion rate, or treatment effect) are treated as fixed but unknown constants.
- Uncertainty comes only from the data (samples), not from the parameters.
In short:
Frequentist = probability as long-run frequency, parameters are fixed.
2. Key Principles
- Probability = Frequency
- Probability of an event = proportion of times it would occur in infinite repetitions.
- Example: “The probability of heads is 0.5” = If we flip the coin many times, ~50% will be heads.
- Parameters are Fixed, Data is Random
- Parameter (e.g., population mean $\mu$) is a fixed, unknown constant.
- The sample mean $\bar{x}$ varies from sample to sample.
- Inference Through Repeated Sampling
- Confidence intervals, p-values, and hypothesis tests are defined based on what would happen if we repeated the experiment many times.
3. Frequentist Tools
- Point Estimation: sample mean $\bar{x}$ estimates true mean $\mu$.
- Confidence Intervals (CIs): intervals that would capture the true parameter in X% of repeated experiments (e.g., 95%).
- Hypothesis Testing:
- Null hypothesis $H_0$.
- p-value = probability of seeing data as extreme or more extreme under $H_0$.
- Maximum Likelihood Estimation (MLE): choose parameter values that maximize likelihood of observed data.
4. Example – A/B Test
- Goal: Does Variant B increase conversion vs Variant A?
- Frequentist approach:
- Parameters $p_A, p_B$ are fixed (true conversion rates).
- Estimate them with sample proportions $\hat{p}_A, \hat{p}_B$.
- Run a two-proportion z-test.
- If p-value < 0.05 → reject $H_0$, conclude B has a different conversion rate.
5. Frequentist vs Bayesian (Contrast)
| Aspect | Frequentist | Bayesian |
|---|---|---|
| Probability | Long-run frequency of events | Degree of belief (subjective) |
| Parameters | Fixed unknown constants | Random variables with distributions |
| Uncertainty | From data (sampling variation) | From both prior + data |
| Inference | p-values, confidence intervals | Posterior probabilities, credible intervals |
| Peeking / Sequential Data | Inflates error (needs corrections) | Allowed, no α inflation |
| Example Statement | “If we repeated this experiment 100 times, 95 of the confidence intervals would contain the true mean.” | “There’s a 95% probability the mean lies between 10 and 15.” |
6. Key Takeaways
- Frequentist statistics = classical framework.
- Probability = frequency, parameters fixed.
- Methods: p-values, hypothesis tests, confidence intervals, MLE.
- Still dominant in many fields (medicine, social sciences, A/B testing).
In short:
The Frequentist approach treats parameters as fixed and probability as long-run frequency. Inference is based on repeated sampling ideas, leading to tools like p-values and confidence intervals.