Informative Prior Distributions

1. The Meaning and Purpose of Informative Priors

An informative prior distribution expresses prior knowledge or beliefs about an unknown parameter before observing data.
In the binomial model, the unknown probability of success is θ. Previously, a uniform prior (where all θ between 0 and 1 are equally likely) was used.

There are two main interpretations of priors:

1. Population interpretation:
   The prior represents a population of possible parameter values, from which the current θ is drawn.
2. Subjective (state-of-knowledge) interpretation:
   The prior expresses our uncertainty about θ, as if θ were a random realization from that prior distribution.

In many real-world settings (for example, estimating the failure probability in a new industrial process), there is no true “population” of θ’s. The prior instead represents all plausible values of θ, and its influence often becomes negligible when the dataset is large.

---

2. The Uniform Prior and Its Justification

A uniform prior for θ implies that all outcomes {0, 1, …, n} in the prior predictive distribution of y are equally likely.

• Bayes’ justification: The uniform prior leads to equal prior predictive probabilities for all observable outcomes.
• Laplace’s justification: Based on the principle of insufficient reason — assign equal probabilities when nothing is known.

However, this principle is not universally valid and can be problematic in complex cases.

---

3. The Beta Prior: A Flexible and Conjugate Family

To include more realistic prior information, we use a Beta(α, β) distribution for θ.

$p(θ) \propto θ^{α – 1}(1 – θ)^{β – 1}$

The likelihood for the binomial model is:

$p(y \mid θ) \propto θ^y (1 – θ)^{n – y}$

Combining prior and likelihood gives the posterior distribution:

$p(θ \mid y) \propto θ^{α + y – 1}(1 – θ)^{β + n – y – 1} = \text{Beta}(α + y, β + n – y)$

This is the conjugate prior for the binomial likelihood — the posterior has the same functional form as the prior.

• Interpretation: α−1 and β−1 can be seen as prior successes and prior failures.

---

Posterior Mean

$E(θ \mid y) = \frac{α + y}{α + β + n}$

This posterior mean always lies between:

• The sample proportion $\frac{y}{n}$​, and
• The prior mean $\frac{α}{α + β}$​.

---

Posterior Variance

$\operatorname{Var}(θ \mid y) = \frac{(α + y)(β + n – y)}{(α + β + n)^2 (α + β + n + 1)}$

As n increases:

$E(θ \mid y) \approx \frac{y}{n}, \quad \operatorname{Var}(θ \mid y) \approx \frac{1}{n} \frac{y}{n}\left(1 – \frac{y}{n}\right)$

Thus, the prior’s influence decreases at the rate of $1/n$.

---

Approximate Normality

By the Central Limit Theorem, for large n:

$\frac{θ – E(θ \mid y)}{\sqrt{\operatorname{Var}(θ \mid y)}} \rightarrow N(0, 1)$

To improve the normal approximation, θ is often transformed using the logit function:

$\text{logit}(θ) = \log\left(\frac{θ}{1 – θ}\right)$

which maps θ ∈ [0, 1] to (−∞, ∞).

---

4. Conjugate Priors and Exponential Families

A class $P$ of prior distributions is conjugate for a class $F$ of likelihoods if:

$p(θ \mid y) \in P \quad \text{for all } p(y \mid θ) \in F \text{ and } p(θ) \in P$

If the likelihood belongs to the exponential family, it has the form:

$p(y_i \mid θ) = f(y_i) g(θ) \exp\left[\phi(θ)^T u(y_i)\right]$

For n independent observations:

$p(y \mid θ) \propto g(θ)^n \exp\left[\phi(θ)^T t(y)\right]$

where the sufficient statistic is $t(y) = \sum_i u(y_i)$.

If the prior is:

$p(θ) \propto g(θ)^{η} \exp\left[\phi(θ)^T ν\right]$

then the posterior is:

$p(θ \mid y) \propto g(θ)^{η + n} \exp\left[\phi(θ)^T (ν + t(y))\right]$

Thus, this prior is conjugate.
Only exponential families have such natural conjugate priors because they have fixed sufficient statistics.

---

5. Example: Probability of a Girl Birth Given Placenta Previa

A study in Germany observed 437 female births out of 980 placenta previa cases.
The general population proportion of female births is 0.485.
Question: Is the probability of a female birth lower among placenta previa cases?

---

(a) Using a Uniform Prior

• Prior: $\text{Beta}(1, 1)$
• Posterior: $\text{Beta}(438, 544)$

Posterior summaries:

$E(θ \mid y) = 0.446, \quad \operatorname{SD}(θ \mid y) = 0.016$ 

$95\% \text{ credible interval: } [0.415, 0.477]$

Simulated 1000 draws from $\text{Beta}(438, 544)$ give:

• Median = 0.446
• 95% interval = [0.415, 0.476]
• Sample mean and SD ≈ exact results.

Logit scale:

$E(\text{logit}(θ) \mid y) = -0.220, \quad \operatorname{SD} = 0.065$

$95\% \text{ interval for } θ: [0.414, 0.477]$

The sex ratio $\frac{1 – θ}{θ}$​ has:

• Posterior median = 1.24
• 95% interval = [1.10, 1.41]
  This is significantly higher than the general population value (1.06).

---

(b) Sensitivity to Different Conjugate Priors

Posterior results were tested for several Beta(α, β) priors centered at 0.485 (population proportion).
As the prior strength $(α + β)$ increases, the posterior mean slightly shifts toward 0.485, but even for $α + β = 200$, the 95% credible interval still excludes 0.485.

→ With a large dataset, the posterior is robust to reasonable prior assumptions.

---

(c) Using a Nonconjugate Prior

A piecewise linear prior centered at 0.485 was used:

• 40% of probability outside [0.385, 0.585]
• Mean = 0.493, SD = 0.21 (similar to Beta with α + β = 5)

Posterior results (computed on a θ-grid):

$\text{Median} = 0.448, \quad 95\% \text{ interval } = [0.419, 0.480]$

Results are nearly identical to the conjugate Beta posterior, confirming that the data dominate the prior when n is large.

---

6. Key Takeaways

• Informative priors encode previous beliefs and can be viewed as “prior data”:
  α−1 prior successes, β−1 prior failures.
• Conjugate priors simplify Bayesian updating and have analytic posteriors.
• Nonconjugate priors allow flexibility but require numerical computation.
• For large samples, posterior inference is insensitive to the prior.
• Exponential-family likelihoods (e.g., binomial, normal, Poisson) always have natural conjugate priors.

---

Summary of Placenta Previa Example

Prior Type	α + β	Prior Mean	Posterior Median	95% Posterior Interval	Interpretation
Uniform (1,1)	2	0.500	0.446	[0.415, 0.477]	Lower female birth rate
Informative near 0.485	20–200	≈0.485	0.447–0.453	[0.416–0.424, 0.478–0.481]	Slight prior influence
Nonconjugate (flat tails)	—	0.493	0.448	[0.419, 0.480]	Similar to conjugate result

---

Conclusion:
The posterior probability of female birth among placenta previa cases is approximately 0.446, lower than 0.485 in the general population — providing strong Bayesian evidence that placenta previa is associated with a lower female birth proportion.