Maximum likelihood is a general method for choosing the parameter values of a probability distribution so that the observed data would be most plausible under that distribution. A concrete example is a set of mouse weights measured in a genetics experiment. The objective is not just to summarize the dataset, but to fit a distribution that can represent any similar experiment of the same type.
1) Why fit a distribution at all?
A fitted distribution is useful because:
- It can be easier to work with than a raw list of measurements.
- It provides a general model that can apply beyond this one dataset—i.e., it can describe what is expected in repeated experiments of the same kind.
- Different types of data naturally match different families of distributions (for example, normal, exponential, gamma, and many others). The distribution family should match the nature of the data.
2) Choosing a distribution family (e.g., Normal)
Suppose mouse weights are believed to be approximately normally distributed. That choice carries expectations:
- Most measurements should be near a central value (the mean). Empirically, most mice do weigh near the dataset’s central tendency.
- The distribution should be roughly symmetric around the mean. The data do not need perfect symmetry; the key is that they are not extremely skewed.
A normal distribution can also vary in spread: it can be “skinny” (small spread) or “wide” (large spread). So choosing “normal” is only part of the task; the parameters still must be determined.
3) Parameters: location and spread
For a normal distribution, the key parameters are:
- Mean (center): $\mu$
- Standard deviation (spread): $\sigma$
Different choices of $\mu$ and $\sigma$ produce different normal distributions. The central MLE question is:
Which values of $\mu$ and $\sigma$ make the observed weights most likely?
4) Likelihood: “How plausible is the observed dataset under this distribution?”
Once a distribution is proposed (say, normal with a certain $\mu$ and $\sigma$), the distribution assigns a probability density to each observed value. The combined plausibility of seeing the entire dataset under those parameters is the likelihood.
For independent measurements $x_1,\dots,x_n$ from a normal distribution, the likelihood is:
$L(\mu,\sigma)=\prod_{i=1}^{n}\frac{1}{\sigma\sqrt{2\pi}}\exp\left(-\frac{(x_i-\mu)^2}{2\sigma^2}\right)$
This is not “the probability that the parameters are true.” Instead, it is a score that says: if the parameters were $\mu$ and $\sigma$, how plausible would it be to observe the data that were actually observed?
5) Intuition for estimating the mean: sliding the curve left and right
Imagine picking an arbitrary normal distribution and overlaying it on the mouse weights. If the distribution’s center $\mu$ is far from where the data cluster, then many observed weights lie in relatively low-density regions of the curve. That makes the likelihood $L(\mu,\sigma)$ small.
Now consider shifting the curve:
- If $\mu$ is moved closer to where the data concentrate (close to the dataset’s average), more points fall in higher-density regions.
- The likelihood increases.
- If the curve keeps shifting past that central region, the likelihood decreases again.
Conceptually, one can think of evaluating the likelihood for many candidate centers $\mu$ and selecting the one that yields the maximum likelihood.
That maximizing center is the maximum likelihood estimate (MLE) of the mean, written $\hat{\mu}$.
For a normal distribution, the result is:
$\hat{\mu}=\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i$
This is an important point: the MLE is an estimate of the distribution’s mean, not merely “the mean of the sample.” It happens that, for the normal distribution, the MLE of the distribution mean equals the sample average.
6) Estimating the standard deviation: adjusting the “width” of the curve
After identifying the best center $\hat{\mu}$, the next job is to find the spread that best matches the data.
Conceptually:
- If $\sigma$ is too small, the distribution is too narrow. Points not extremely close to $\hat{\mu}$ receive very low density, reducing likelihood.
- If $\sigma$ is too large, the curve becomes too flat. The density near the center also becomes lower than it needs to be, again reducing likelihood.
So the likelihood can be evaluated over different $\sigma$ values, and the $\sigma$ that maximizes it is selected as $\hat{\sigma}$.
For the normal distribution, the MLE of variance is:
$\hat{\sigma}^2=\frac{1}{n}\sum_{i=1}^{n}(x_i-\hat{\mu})^2$
and therefore:
$\hat{\sigma}=\sqrt{\frac{1}{n}\sum_{i=1}^{n}(x_i-\hat{\mu})^2}$
Note the denominator is $n$, not $n-1$. The $n-1$ version is a different estimator (the unbiased sample variance), but MLE for the normal uses $n$.
7) What “maximum likelihood estimates” means in plain terms
When someone says they found maximum likelihood estimates for parameters (mean, standard deviation, or anything else), it means:
- A distribution family was chosen (normal, exponential, gamma, etc.).
- Parameters were treated as unknown.
- Parameter values were selected to maximize the likelihood of observing the actual measurements.
In short:
MLE picks the parameter values that make the observed data most plausible under the assumed distribution.
8) “Probability” vs. “likelihood” (terminology clarification)
In everyday speech, “probability” and “likelihood” are often used interchangeably. In statistics, “likelihood” has a specific meaning:
- Probability typically treats parameters as fixed and the data as random.
- Likelihood treats the observed data as fixed and views the parameters as variables to be tuned.
MLE is the classic “likelihood” situation: given observed measurements, choose parameter values that maximize $L(\text{parameters})$.
9) Final outcome: the fitted distribution
After computing $\hat{\mu}$ and $\hat{\sigma}$, the fitted model is the normal distribution:
$X\sim \mathcal{N}(\hat{\mu},\hat{\sigma}^2)$
This fitted distribution is then used as a general representation for mouse weights in similar experiments, enabling inference, prediction, and comparison in a principled way.