1. Setup

We model data from n independent Bernoulli trials (success/failure events) with probability of success $p$.

  • $n$ = number of trials
  • $k$ = number of observed successes
  • $p$ = probability of success (parameter of interest)

2. Binomial Probability Mass Function (PMF)

The probability of observing exactly $k$ successes in $n$ trials:

$P(X = k \mid p) = {n \choose k} p^k (1-p)^{n-k}$

3. Likelihood Function

The likelihood treats the parameter $p$ as the variable (data $k$ and $n$ are fixed).

$L(p \mid k, n) = {n \choose k} p^k (1-p)^{n-k}$

  • Same formula as PMF, but now seen as a function of $p$.
  • The constant ${n \choose k}$ does not depend on $p$, so often omitted in inference:

$L(p \mid k, n) \propto p^k (1-p)^{n-k}$

4. Maximum Likelihood Estimation (MLE)

  • To estimate $p$, maximize the likelihood.
  • Equivalent to maximizing:

$\ell(p) = k \ln(p) + (n-k)\ln(1-p)$

  • Derivative set to zero:

$\hat{p}_{MLE} = \frac{k}{n}$

Intuition: the MLE for $p$ is just the observed proportion of successes.

5. Example

  • $n = 10$, $k = 7$.
  • Likelihood:

$L(p) \propto p^7 (1-p)^3$

  • Plot: bell-shaped, peaked at $p = 0.7$.
  • MLE: $\hat{p} = 7/10 = 0.7$.

6. Bayesian Inference with Binomial Likelihood

  • If the prior on $p$ is Beta($\alpha, \beta$),
  • Posterior is also Beta($\alpha+k, \beta+n-k$) (conjugacy).

This makes binomial likelihood + beta prior a classic pair in Bayesian statistics.

7. Key Takeaways

  • Binomial likelihood: likelihood of parameter $p$ given $k$ successes in $n$ trials.
  • Formula: $L(p) \propto p^k (1-p)^{n-k}$.
  • MLE = sample proportion $\hat{p} = k/n$.
  • In Bayesian inference, with Beta prior, posterior is also Beta → easy updating.

In short:
The binomial likelihood expresses how likely the observed number of successes $k$ is for a given success probability $p$. It has the form $L(p) \propto p^k (1-p)^{n-k}$, peaks at the observed proportion $k/n$, and is central in both frequentist (MLE) and Bayesian inference (Beta-Binomial model).