Beta Distribution

1. Definition

• The Beta distribution is a continuous probability distribution defined on the interval $[0,1]$.
• It is controlled by two shape parameters: $\alpha$ and $\beta$.
• Probability density function (PDF):

$f(p \mid \alpha, \beta) = \frac{1}{B(\alpha, \beta)} \; p^{\alpha – 1} (1-p)^{\beta – 1}, \quad 0 \leq p \leq 1$

where

• $B(\alpha, \beta)$ = Beta function = normalizing constant.

---

2. Key Properties

• Support: $[0,1]$ → often used to model probabilities or proportions.
• Mean:

$E[p] = \frac{\alpha}{\alpha + \beta}$

• Variance:

$Var(p) = \frac{\alpha \beta}{(\alpha + \beta)^2 (\alpha + \beta + 1)}$

• Mode (if $\alpha, \beta > 1$):

$\text{Mode} = \frac{\alpha – 1}{\alpha + \beta – 2}$

---

3. Shapes of Beta Distributions

Depending on $\alpha$ and $\beta$:

• $\alpha = \beta = 1$: Uniform(0,1).
• $\alpha > \beta$: Skewed toward 1.
• $\alpha < \beta$: Skewed toward 0.
• Both $\alpha, \beta > 1$: Unimodal distribution.
• Both $\alpha, \beta < 1$: U-shaped (peaks near 0 and 1).

---

4. Connection to Binomial Likelihood (Conjugacy)

• The Beta distribution is the conjugate prior for the Binomial likelihood.
• Prior: $p \sim \text{Beta}(\alpha, \beta)$.
• Data: $k$ successes in $n$ trials.
• Posterior:

$p \mid \text{data} \sim \text{Beta}(\alpha + k, \; \beta + n – k)$

This makes Bayesian updating simple.

---

5. Examples

(a) Uniform Prior

• Beta(1,1) → flat prior on [0,1].
• No preference before data.

(b) Strong Belief Around 0.5

• Beta(20,20) → very peaked near 0.5.
• Encodes strong prior belief in fairness (like a fair coin).

(c) After Observing Data

• Prior: Beta(1,1).
• Data: 7 heads, 3 tails.
• Posterior: Beta(1+7, 1+3) = Beta(8,4).
• Mean = $8 / (8+4) = 0.67$.
  Updated belief about $p$ after observing data.

---

6. Applications

• Bayesian Inference: posterior distributions of probabilities.
• A/B Testing: posterior belief about conversion rates.
• Machine Learning: Dirichlet distribution (generalization of Beta).
• Reliability analysis: modeling success/failure rates.
• Proportion modeling: when outcomes are percentages or rates.

---

7. Key Takeaways

• The Beta distribution is a flexible family of distributions on $[0,1]$.
• Controlled by shape parameters $\alpha$ and $\beta$.
• Conjugate prior for the Binomial likelihood.
• Central in Bayesian updating for probabilities (like coin tosses, conversion rates).

---

In short:
The Beta distribution is a probability distribution on $[0,1]$, parameterized by $\alpha, \beta$. It’s the conjugate prior for the binomial likelihood, making it essential in Bayesian inference about probabilities (e.g., coin bias, conversion rates).