The Normal Distribution, Empirical Rule, and Normal Approximation
Many statistical methods use the normal distribution as a mathematical model for variation. Its familiar bell-shaped curve appears when observations cluster around a central value, become less common farther from the center, and are approximately symmetric.
However, not every dataset is normally distributed. Before applying a normal model, the shape and context of the data should be examined.
Bell-Shaped Data
A bell-shaped histogram usually has the following features:
- one central peak;
- approximate symmetry;
- decreasing frequency away from the center;
- relatively thin tails;
- few extremely distant observations.
A conceptual bell-shaped histogram looks like:
Frequency | | █ | █████████ | ███████████████ | █████████████████████ | ███████████████████████████ +------------------------------------ Value Center
Examples that may be approximately bell-shaped within carefully defined populations include:
- adult height;
- measurement errors;
- standardized test scores;
- repeated laboratory measurements;
- some biological characteristics.
Whether a variable is approximately normal depends on the population, measurement process, and subgroup definition.
Historical Height Data
A well-known historical dataset collected by Francis Galton contains measurements of parents and their adult children. A histogram of the heights of 928 fathers is often used as an example of an approximately bell-shaped distribution.
Most measured heights occur near the center, with fewer observations at very short or very tall values.
This does not mean that human height is exactly normal. Real height data may differ because of:
- age;
- sex;
- nutrition;
- ancestry;
- measurement error;
- population mixture;
- sampling procedures.
The normal curve is an approximation to the distribution, not a literal description of every observation.
Histogram Versus Normal Curve
A histogram summarizes observed data. A normal curve is a continuous mathematical model.
Histogram
A histogram depends on:
- the observed sample;
- sample size;
- bin width;
- bin boundaries;
- measurement precision.
Normal curve
A normal curve is determined by two parameters:
- mean, \(\mu\);
- standard deviation, \(\sigma\).
It represents a theoretical probability distribution rather than a set of bars.
A normal curve may be overlaid on a histogram to evaluate whether the model provides a reasonable approximation.
Data do not become normal merely because a bell-shaped curve is drawn over their histogram.
The Normal Distribution
A random variable \(X\) follows a normal distribution with mean \(\mu\) and standard deviation \(\sigma\) when its probability density is:\[ f(x) = \frac{1}{ \sigma\sqrt{2\pi} } \exp \left[ -\frac{(x-\mu)^2}{2\sigma^2} \right]. \]
This is written:\[ \boxed{ X\sim N(\mu,\sigma^2) } \]
where:
- \(\mu\) determines the center;
- \(\sigma\) determines the spread;
- \(\sigma^2\) is the variance.
The formula is less important than the curve’s structural properties.
Properties of the Normal Curve
A normal distribution is:
- continuous;
- symmetric around \(\mu\);
- unimodal;
- completely determined by \(\mu\) and \(\sigma\);
- asymptotic to the horizontal axis;
- equal in mean, median, and mode.
For a normal distribution:\[ \text{mean} = \text{median} = \text{mode} = \mu. \]
The total area under the curve is:\[ \int_{-\infty}^{\infty}f(x)\,dx=1. \]
Probabilities correspond to areas under the curve.
The Mean Controls Location
Changing \(\mu\) moves the curve horizontally without changing its basic shape.
Smaller mean Larger mean /\ /\ / \ / \_____/ \_____ _______/ \_____
If:\[ X_1\sim N(50,10^2) \]
and:\[ X_2\sim N(80,10^2), \]
the two distributions have the same spread but different centers.
The Standard Deviation Controls Spread
Changing \(\sigma\) changes the width of the curve.
Small standard deviation Large standard deviation /\ __ / \ _/ \_________/ \________ ____/ \____
A smaller \(\sigma\) produces a narrower and taller curve. A larger \(\sigma\) produces a wider and shorter curve.
The total area remains 1 in both cases.
Density Is Not Probability at a Point
For a continuous random variable:\[ P(X=x)=0 \]
for every exact value \(x\).
Probability is assigned to intervals:\[ P(a\leq X\leq b) = \int_a^b f(x)\,dx. \]
The height \(f(x)\) is a probability density, not a probability by itself.
A tall region indicates greater probability concentration per unit of the horizontal scale. The probability over an interval is determined by area.
The Empirical Rule
For an approximately normal distribution:
- about 68% of observations fall within one standard deviation of the mean;
- about 95% fall within two standard deviations;
- about 99.7% fall within three standard deviations.
Symbolically:\[ \boxed{ P(\mu-\sigma\leq X\leq\mu+\sigma) \approx0.68 } \]\[ \boxed{ P(\mu-2\sigma\leq X\leq\mu+2\sigma) \approx0.95 } \]\[ \boxed{ P(\mu-3\sigma\leq X\leq\mu+3\sigma) \approx0.997 } \]
This is called the empirical rule or the 68–95–99.7 rule.
Empirical Rule Diagram
Normal curve
μ
|
μ−σ | μ+σ
| | |
______|________|__________|______
/ \
/ \
/ \
About 68% within 1σ
μ−2σ μ+2σ
| |
____|__________________________________|____
About 95% within 2σ
μ−3σ μ+3σ
| |
____|____________________________________________|____
About 99.7% within 3σ
Because the normal distribution is symmetric, the remaining probability is divided equally between the two tails.
Tail Areas from the Empirical Rule
If about 68% lies within one standard deviation, then:\[ 100\%-68\%=32\% \]
lies outside that interval.
By symmetry, each tail contains approximately:\[ \frac{32\%}{2}=16\%. \]
Thus:\[ P(X>\mu+\sigma)\approx0.16. \]
Similarly, if 95% lies within two standard deviations:\[ 100\%-95\%=5\%. \]
Each tail contains approximately:\[ 2.5\%. \]
Therefore:\[ P(X>\mu+2\sigma)\approx0.025. \]
For three standard deviations:\[ 100\%-99.7\%=0.3\%, \]
so each tail contains approximately:\[ 0.15\%. \]
Example Using the Empirical Rule
Suppose adult heights in a defined population are approximately normal with:\[ \mu=175\text{ cm} \]
and:\[ \sigma=7\text{ cm}. \]
Within one standard deviation
\[ 175-7=168 \]
and:\[ 175+7=182. \]
Approximately 68% of heights fall between:\[ 168\text{ cm and }182\text{ cm}. \]
Within two standard deviations
\[ 175-2(7)=161 \]
and:\[ 175+2(7)=189. \]
Approximately 95% fall between:\[ 161\text{ cm and }189\text{ cm}. \]
Within three standard deviations
\[ 175-3(7)=154 \]
and:\[ 175+3(7)=196. \]
Approximately 99.7% fall between:\[ 154\text{ cm and }196\text{ cm}. \]
These calculations are appropriate only if the normal model is a reasonable approximation for the specified population.
Z-Scores
A z-score measures how many standard deviations an observation lies above or below the mean.
For a value \(x\):\[ \boxed{ z=\frac{x-\mu}{\sigma} } \]
Interpretation:
- \(z=0\): exactly at the mean;
- \(z=1\): one standard deviation above the mean;
- \(z=-1\): one standard deviation below the mean;
- \(z=2\): two standard deviations above the mean.
For sample data, a standardized value is often calculated as:\[ z=\frac{x-\bar{x}}{s}. \]
Z-Score Example
Using:\[ \mu=175 \]
and:\[ \sigma=7, \]
consider a height of 189 cm:\[ z = \frac{189-175}{7} = 2. \]
The observation is two standard deviations above the mean.
Under a normal model, approximately 2.5% of observations lie above this point.
Consider 168 cm:\[ z = \frac{168-175}{7} = -1. \]
This is one standard deviation below the mean.
The Standard Normal Distribution
A normally distributed variable can be standardized.
If:\[ X\sim N(\mu,\sigma^2), \]
then:\[ Z=\frac{X-\mu}{\sigma} \]
has the standard normal distribution:\[ \boxed{ Z\sim N(0,1) } \]
The standard normal distribution has:\[ \mu_Z=0 \]
and:\[ \sigma_Z=1. \]
This transformation allows probabilities for any normal distribution to be calculated using one common curve.
Why Normal Approximation Is Useful
The normal distribution is important for two separate reasons.
Some observed distributions are approximately normal
Certain measurements produce roughly symmetric, bell-shaped histograms.
Some statistics have approximately normal sampling distributions
Even when the original data are not normal, quantities such as sample means and sample proportions may have approximately normal sampling distributions under suitable conditions.
This second role is especially important because it supports:
- confidence intervals;
- hypothesis tests;
- margin-of-error calculations;
- approximations to discrete distributions.
Random Variables
A random variable assigns a numerical value to each outcome of a random experiment.
For three die rolls, define:\[ X=\text{the number of sixes}. \]
Then:\[ X\in\{0,1,2,3\}. \]
The exact value of \(X\) is unknown before the dice are rolled, but its possible values and probabilities can be described.
Random variables may be:
- discrete;
- continuous.
Discrete Random Variables
A discrete random variable takes values from a countable set.
Examples include:
- number of sixes in ten rolls;
- number of defective products;
- number of customers arriving;
- number of positive test results;
- number of spam messages.
A probability mass function assigns probabilities:\[ P(X=x). \]
These probabilities satisfy:\[ P(X=x)\geq0 \]
and:\[ \sum_xP(X=x)=1. \]
Continuous Random Variables
A continuous random variable can take values across an interval.
Examples include:
- height;
- weight;
- time;
- temperature;
- distance;
- blood pressure.
Continuous distributions use probability density functions. Probabilities correspond to areas:\[ P(a\leq X\leq b) = \int_a^bf(x)\,dx. \]
The normal distribution is continuous.
The Binomial Distribution
The binomial distribution models the number of successes in a fixed number of independent trials when each trial has the same success probability.
Let:\[ X=\text{number of successes in }n\text{ trials}. \]
Then:\[ \boxed{ X\sim\operatorname{Binomial}(n,p) } \]
when:
- the number of trials \(n\) is fixed;
- every trial has two outcomes, called success and failure;
- trials are independent;
- success probability \(p\) is constant.
The probability of exactly \(k\) successes is:\[ \boxed{ P(X=k) = \binom{n}{k} p^k(1-p)^{n-k} } \]
for:\[ k=0,1,\ldots,n. \]
Mean and Standard Deviation of a Binomial Variable
If:\[ X\sim\operatorname{Binomial}(n,p), \]
then:\[ E[X]=np \]
and:\[ \operatorname{Var}(X)=np(1-p). \]
Therefore:\[ \boxed{ \operatorname{SD}(X) = \sqrt{np(1-p)} } \]
For the sample proportion:\[ \hat{p}=\frac{X}{n}, \]
we have:\[ E[\hat{p}]=p \]
and:\[ \operatorname{SD}(\hat{p}) = \sqrt{ \frac{p(1-p)}{n} }. \]
Binomial Example
Suppose a fair coin is tossed 100 times:\[ X=\text{number of heads}. \]
Then:\[ X\sim\operatorname{Binomial}(100,0.5). \]
The expected number of heads is:\[ E[X] = 100(0.5) = 50. \]
The standard deviation is:\[ \operatorname{SD}(X) = \sqrt{ 100(0.5)(0.5) } = 5. \]
The empirical-rule intuition suggests that the number of heads will often fall within approximately:\[ 50\pm2(5), \]
or:\[ 40\text{ to }60, \]
because this range is two standard deviations from the mean.
The exact binomial probability and the normal approximation are not identical, but the approximation can be useful.
Normal Approximation to the Binomial
For sufficiently large \(n\), a binomial distribution can be approximated by a normal distribution with:\[ \mu=np \]
and:\[ \sigma=\sqrt{np(1-p)}. \]
Thus:\[ X \approx N\left( np, np(1-p) \right). \]
A common guideline is that both expected counts are sufficiently large:\[ np\geq10 \]
and:\[ n(1-p)\geq10. \]
Different texts use different thresholds. These are practical guidelines, not universal laws.
The approximation tends to be poor when:
- \(n\) is small;
- \(p\) is very close to 0;
- \(p\) is very close to 1;
- the distribution is strongly skewed.
Continuity Correction
The binomial distribution is discrete, while the normal distribution is continuous.
To approximate:\[ P(X\leq k), \]
use:\[ P(X\leq k) \approx P(Y\leq k+0.5), \]
where:\[ Y\sim N(np,np(1-p)). \]
To approximate:\[ P(X=k), \]
use:\[ P(k-0.5<Y<k+0.5). \]
This adjustment is called the continuity correction and often improves the approximation.
When Data Are Not Normal
Many important variables have non-normal distributions.
Examples include:
- income;
- wealth;
- home prices;
- hospital stays;
- transaction values;
- insurance claims;
- online engagement;
- waiting times.
These distributions may be:
- strongly skewed;
- heavy-tailed;
- multimodal;
- bounded;
- dominated by zeros;
- affected by extreme observations.
For such data, the empirical rule may be misleading.
For example, income distributions typically have a long right tail. The mean and standard deviation may be heavily influenced by a small number of very large values.
Median, quartiles, IQR, percentiles, or transformations may be more informative.
Mixtures Can Hide Subgroups
Even when each subgroup is approximately normal, combining them can produce a non-normal distribution.
Suppose two populations have:\[ X_1\sim N(\mu_1,\sigma_1^2) \]
and:\[ X_2\sim N(\mu_2,\sigma_2^2). \]
If their means differ substantially, the combined histogram may be bimodal.
A histogram with two peaks may indicate:
- different subpopulations;
- different measurement conditions;
- changes over time;
- an omitted categorical variable.
A single normal curve may then conceal important structure.
Checking Whether a Normal Model Is Reasonable
Histogram
Look for:
- approximate symmetry;
- one central peak;
- tails that decrease gradually;
- absence of extreme gaps or clusters.
Boxplot
Check for:
- a median near the center of the box;
- approximately balanced whiskers;
- few extreme observations.
Q–Q plot
A normal quantile–quantile plot compares ordered observations with theoretical normal quantiles.
If the data are approximately normal, points lie near a straight line.
Systematic curvature may indicate:
- skewness;
- heavy tails;
- light tails;
- outliers.
No graph proves exact normality. The goal is to determine whether the approximation is adequate for the intended use.
Empirical Rule Versus Chebyshev’s Inequality
The empirical rule relies on an approximately normal shape.
For an arbitrary distribution with finite variance, Chebyshev’s inequality states:\[ P(|X-\mu|<k\sigma) \geq 1-\frac{1}{k^2} \]
for:\[ k>1. \]
For \(k=2\):\[ P(|X-\mu|<2\sigma) \geq 1-\frac{1}{4} = 75\%. \]
For \(k=3\):\[ P(|X-\mu|<3\sigma) \geq 1-\frac{1}{9} \approx88.9\%. \]
Compare this with the stronger normal-model values:
| Interval | Any distribution under Chebyshev | Normal distribution |
|---|---|---|
| Within \(2\sigma\) | At least 75% | About 95% |
| Within \(3\sigma\) | At least 88.9% | About 99.7% |
Chebyshev’s inequality is more general but less precise.
Common Misconceptions
“Many datasets are exactly normal”
Real data rarely follow a theoretical distribution exactly. Normality is usually an approximation.
“Every bell-shaped histogram is normal”
Other symmetric distributions can also appear bell-shaped.
“The empirical rule applies to all data”
It requires an approximately normal distribution.
“A value beyond three standard deviations must be an error”
It is unusual under a normal model, but it may be a valid observation or evidence that the model is inappropriate.
“A histogram determines normality conclusively”
Histogram appearance depends on sample size and bin selection.
“If the raw data are skewed, no normal method can be used”
Some sample statistics can have approximately normal sampling distributions even when the original observations are skewed, provided suitable conditions hold.
“IQ scores independently prove a natural normal law”
Standardized scores are often designed or transformed to have a specified mean and standard deviation, so their shape partly reflects the construction process.
A Practical Analysis Process
- Identify whether the variable is discrete or continuous.
- Plot the distribution.
- examine symmetry, peaks, tails, and unusual observations.
- calculate the mean, median, standard deviation, and IQR.
- compare the mean and median.
- consider whether subgroups have been combined.
- use a Q–Q plot when normality matters.
- apply the empirical rule only when the approximation is reasonable.
- check binomial approximation conditions for count data.
- report limitations of the model.
Key Takeaway
The normal distribution is a symmetric, bell-shaped probability model determined by its mean and standard deviation. For approximately normal data, about 68%, 95%, and 99.7% of observations lie within one, two, and three standard deviations of the mean. The normal curve can describe certain measurements and approximate sampling distributions or binomial counts under suitable conditions. It should not be applied automatically: skewness, heavy tails, multiple subgroups, extreme values, and small expected counts can make the approximation unreliable.