Posterior

1. Definition

• The posterior is the updated probability distribution of a parameter (or hypothesis) after observing data.
• It represents the new belief after combining:
  • the prior belief (what we thought before), and
  • the likelihood (evidence from data).

In short:

Posterior = Prior updated with Evidence.

---

2. Formula (Bayes’ Theorem)

For a parameter $\theta$ and data $D$:

$P(\theta \mid D) = \frac{P(D \mid \theta) \cdot P(\theta)}{P(D)}$

Where:

• $P(\theta)$ = prior (belief before data)
• $P(D \mid \theta)$ = likelihood (probability of data given parameter)
• $P(D)$ = marginal likelihood (normalizing constant)
• $P(\theta \mid D)$ = posterior (updated belief)

---

3. Interpretation

• The posterior distribution shows which parameter values are most plausible after observing the data.
• It allows statements like:
  • “There’s a 95% probability that the conversion rate is between 4% and 6%.”
  • This is different from frequentist confidence intervals, which don’t allow direct probability statements about parameters.

---

4. Example – Coin Toss

We want to estimate the probability of heads (ppp).

• Prior: Uniform prior = Beta(1,1) → all values of $p \in [0,1]$ equally likely.
• Data: 10 tosses, 7 heads.
• Likelihood: Binomial(10, p).
• Posterior:

$P(p \mid \text{data}) \propto p^7 (1-p)^3 \cdot \text{Prior}(p)$

Since prior = Beta(1,1), posterior = Beta(1+7, 1+3) = Beta(8,4).

Posterior distribution: mean ≈ 0.67 → belief updated toward p ~ 0.67 after data.

---

5. Posterior vs Posterior Probability

• Posterior distribution: Full probability distribution of parameter after data.
• Posterior probability: Probability that a hypothesis/parameter lies in a certain region (e.g., P(p > 0.5 | data) = 0.9).

---

6. Applications

• A/B testing: posterior probability that variant B > A.
• Medicine: probability that treatment works after observing trial data.
• Machine learning: Bayesian neural networks, Naive Bayes.
• Forecasting: update predictions with new data.

---

7. Key Takeaways

• Posterior = updated belief about parameters/hypotheses after seeing data.
• Formula: Posterior ∝ Prior × Likelihood.
• More data → posterior is dominated by likelihood (prior matters less).
• Core of Bayesian inference → all decisions are based on posterior.

---

In short:
The posterior is the probability distribution of a parameter after observing data. It combines prior belief with the likelihood of observed evidence, giving the updated belief used in Bayesian inference.