Likelihood Ratio (LR)

1. Definition

• The likelihood ratio (LR) is a measure of how much more likely the observed data is under one hypothesis compared to another.
• It compares the likelihood functions of two competing hypotheses.

$\Lambda = \frac{L(\text{data} \mid H_1)}{L(\text{data} \mid H_0)}$

Where:

• $L(\text{data} \mid H)$ = probability (or probability density) of the observed data given hypothesis H.
• $H_0$​ = null hypothesis.
• $H_1$​ = alternative hypothesis.

Intuition:

• LR = 1 → data is equally likely under both hypotheses.
• LR > 1 → data favors $H_1$​.
• LR < 1 → data favors $H_0$​.

---

2. Likelihood Ratio in Hypothesis Testing

(a) Likelihood Ratio Test (LRT)

• Use LR to decide whether to reject $H_0$​.
• Test statistic:

$\Lambda = \frac{\sup_{\theta \in \Theta_0} L(\theta)}{\sup_{\theta \in \Theta} L(\theta)}$

Where:

• Numerator = maximum likelihood under null hypothesis space.
• Denominator = maximum likelihood under full parameter space.
• Small $\Lambda$ → strong evidence against $H_0$​.

---

(b) Sequential Probability Ratio Test (SPRT)

• Update LR as new data arrives:

$\Lambda_n = \frac{L(\text{data up to n} \mid H_1)}{L(\text{data up to n} \mid H_0)}$

• Compare against thresholds:
  • If $\Lambda_n > A$ → accept $H_1$​.
  • If $\Lambda_n < B$ → accept $H_0$​.
  • Else → continue sampling.

---

3. Example – Coin Toss

Suppose we flip a coin 10 times, and get 7 heads.

• $0H_0$​: coin is fair (p = 0.5).
• $H_1$​: coin is biased (p = 0.7).

Likelihood under $H_0$​:

$L(H_0) = {10 \choose 7} (0.5)^7 (0.5)^3 = 120 \times 0.5^{10} \approx 0.117$

Likelihood under $H_1$​:

$L(H_1) = {10 \choose 7} (0.7)^7 (0.3)^3 \approx 0.266$

Likelihood Ratio:

$\Lambda = \frac{0.266}{0.117} \approx 2.27$

Interpretation: The data is about 2.3 times more likely under $H_1$​ than under $H_0$​.

---

4. Interpretation Guidelines (often used in practice)

• $\Lambda \approx 1$ → no evidence.
• $\Lambda > 3$ → moderate evidence for $H_1$​.
• $\Lambda > 10$ → strong evidence for $H_1$​.
• (Symmetrically, very small $\Lambda$ favors $H_0$​).

---

5. Applications

• Classical hypothesis testing → Likelihood Ratio Test.
• Sequential tests → SPRT.
• Model comparison → nested models.
• Bayesian inference → Bayes factors are closely related to likelihood ratios.
• Medical testing → likelihood ratios of diagnostic tests (LR+ and LR–).

---

6. Key Takeaways

• A likelihood ratio is the relative likelihood of data under two hypotheses.
• Central to likelihood-based inference (LRT, SPRT, Bayes factors).
• Higher LR → stronger evidence for $H_1$​; lower LR → stronger evidence for $H_0$​.

---

In short:
The likelihood ratio (LR) compares how well two hypotheses explain the observed data. If LR > 1, the data supports $H_1$1​; if LR < 1, it supports $H_0$​. It’s the backbone of many statistical tests, including the Likelihood Ratio Test and Sequential Probability Ratio Test.