Posterior Probability

1. Definition

• The posterior probability is the updated probability of a hypothesis (or parameter value) after observing data, using Bayes’ theorem.
• It combines:
  1. Prior probability (belief before seeing data), and
  2. Likelihood (evidence from data).

Formula:

$P(H \mid \text{data}) = \frac{P(\text{data} \mid H) \cdot P(H)}{P(\text{data})}$

Where:

• $P(H)$ = prior probability of hypothesis H
• $P(\text{data} \mid H)$ = likelihood of the observed data given H
• $P(\text{data})$ = marginal probability of data (normalizing constant)

---

2. Interpretation

• Posterior probability answers:
  “Given the data I’ve observed, how probable is this hypothesis (or parameter value)?”
• Very different from frequentist p-values, which answer:
  “If H₀ were true, how likely is the data (or more extreme)?”

---

3. Example – Coin Toss

Suppose we want to infer if a coin is biased toward heads.

• Hypotheses:
  • $H_0$: coin is fair (p = 0.5)
  • $H_1$​: coin is biased (p = 0.7)
• Prior: $P(H_0) = P(H_1) = 0.5$ (equal belief).
• Data: 10 tosses, 7 heads.

Likelihoods:

• $P(\text{data} \mid H_0) = {10 \choose 7} (0.5)^{10} ≈ 0.117$
• $P(\text{data} \mid H_1) = {10 \choose 7} (0.7)^7 (0.3)^3 ≈ 0.266$

Posterior for H₁:

$P(H_1 \mid \text{data}) = \frac{0.266 \cdot 0.5}{0.266 \cdot 0.5 + 0.117 \cdot 0.5} = \frac{0.133}{0.133 + 0.0585} \approx 0.694$

Interpretation: After seeing 7 heads in 10 tosses, the probability that the coin is biased is about 69%.

---

4. Posterior Distribution (Parameters)

• If we estimate parameters (not just discrete hypotheses), the posterior becomes a distribution:

$P(\theta \mid \text{data}) \propto P(\text{data} \mid \theta) \cdot P(\theta)$

Where:

• $P(\theta)$ = prior distribution
• $P(\text{data} \mid \theta)$ = likelihood
• $P(\theta \mid \text{data})$ = posterior distribution

This is used for Bayesian inference, e.g., estimating means, regression coefficients, conversion rates.

---

5. Applications

• Bayesian sequential testing: stop experiment when posterior $P(H_1 \mid data)$ > 0.95.
• Bayesian estimation: use posterior distribution instead of point estimates.
• Machine learning: Bayesian models (Naive Bayes, Bayesian neural nets).
• Medical decisions: posterior probability of disease given test result.

---

6. Key Takeaways

• Posterior probability = probability of a hypothesis given observed data.
• Formula: Prior × Likelihood → normalized.
• Provides a direct, intuitive interpretation (unlike p-values).
• In Bayesian inference, all decisions are based on the posterior.

---

In short:
The posterior probability is the updated probability of a hypothesis or parameter after seeing data. It’s calculated using Bayes’ theorem and combines prior belief with evidence from data.