Standard Error of the Sample Mean and the Square-Root Law
A randomly selected observation varies around the population mean. A sample mean also varies, but it is generally more stable than an individual observation. The standard error measures this sample-to-sample variability.
Individual Observations from a Population
Suppose adult male height in a population has:
- Population mean \(\mu\)
- Population standard deviation \(\sigma\)
Let \(X\) be the height of one randomly selected adult male. Then\[ E(X)=\mu \]
and\[ SD(X)=\sigma. \]
The expected value \(\mu\) identifies the center of the population distribution, while \(\sigma\) measures the spread of individual heights around that center.
If the height distribution is approximately normal, about 68% of individual heights fall within one standard deviation of the mean:\[ \mu-\sigma\leq X\leq\mu+\sigma. \]
However, this 68% interpretation is specific to approximately normal distributions. For an arbitrary population, \(\sigma\) remains a measure of spread, but it does not guarantee that most observations fall within exactly one standard deviation of the mean.
From One Observation to a Sample Mean
Suppose we independently select \(n\) individuals and record their heights:\[ X_1,X_2,\ldots,X_n. \]
The sample mean is\[ \bar{X}_n = \frac{X_1+X_2+\cdots+X_n}{n}. \]
The subscript \(n\) emphasizes that the statistic is based on a sample of size \(n\). It is often omitted when the sample size is clear:\[ \bar{X}=\bar{X}_n. \]
Although \(\bar{X}\) is an average, it is still random before the sample is selected. Different random samples generally produce different sample means.
For example, if the population mean is\[ \mu=69.3\text{ inches}, \]
one sample might produce\[ \bar{X}=70.1\text{ inches}, \]
while another might produce\[ \bar{X}=69.1\text{ inches}. \]
The population mean remains fixed at 69.3 inches. It is the sample mean that varies.
Expected Value of the Sample Mean
The expected value of the sample mean is\[ E(\bar{X}) = E\left( \frac{X_1+\cdots+X_n}{n} \right). \]
Using the linearity of expectation,\[ E(\bar{X}) = \frac{E(X_1)+\cdots+E(X_n)}{n}. \]
If every observation is drawn from the same population,\[ E(X_i)=\mu. \]
Therefore,\[ E(\bar{X}) = \frac{n\mu}{n} = \mu. \]
Thus,\[ \boxed{E(\bar{X})=\mu}. \]
The sample mean is therefore an unbiased estimator of the population mean.
Unbiasedness means that the average of the sample means over repeated random samples equals the population mean. It does not mean that every individual sample mean equals \(\mu\).
What Is a Standard Error?
The standard error of a statistic is the standard deviation of its sampling distribution.
For the sample mean,\[ SE(\bar{X}) = SD(\bar{X}). \]
It measures how much sample means vary across repeated random samples of the same size.
This creates an important distinction:
| Quantity | What varies? | What it measures |
|---|---|---|
| Population standard deviation \(\sigma\) | Individual observations | Spread of individual values |
| Standard error \(SE(\bar{X})\) | Sample means | Spread of sample averages |
The standard deviation answers:
How much do individual heights differ from the population mean?
The standard error answers:
How much would the sample mean differ across repeated samples?
The Square-Root Law
If \(X_1,\ldots,X_n\) are independent observations from a population with standard deviation \(\sigma\), then\[ \operatorname{Var}(\bar{X}) = \operatorname{Var} \left( \frac{X_1+\cdots+X_n}{n} \right). \]
Since variance scales with the square of a constant,\[ \operatorname{Var}(\bar{X}) = \frac{1}{n^2} \operatorname{Var}(X_1+\cdots+X_n). \]
Under independence, variances add:\[ \operatorname{Var}(X_1+\cdots+X_n) = n\sigma^2. \]
Therefore,\[ \operatorname{Var}(\bar{X}) = \frac{n\sigma^2}{n^2} = \frac{\sigma^2}{n}. \]
Taking the square root gives\[ \boxed{ SE(\bar{X}) = \frac{\sigma}{\sqrt{n}} }. \]
This relationship is known as the square-root law.
Why Averaging Reduces Variability
Individual observations contain random variation. When independent observations are averaged, values above the population mean tend to be offset by values below it.
The expected value remains unchanged:\[ E(\bar{X})=\mu, \]
but the standard error becomes smaller:\[ SE(\bar{X})=\frac{\sigma}{\sqrt{n}}. \]
Therefore, a sample mean is generally more stable than a single observation.
For one observation,\[ SE(X)=\sigma. \]
For four observations,\[ SE(\bar{X})=\frac{\sigma}{2}. \]
For 100 observations,\[ SE(\bar{X})=\frac{\sigma}{10}. \]
Example with Heights
Suppose adult male height in a population has\[ \mu=69.3\text{ inches} \]
and\[ \sigma=3.0\text{ inches}. \]
One Randomly Selected Individual
For one individual,\[ E(X)=69.3 \]
and\[ SD(X)=3.0. \]
An individual’s height typically varies on a scale of approximately 3 inches around the population mean.
Sample of 25 Individuals
For \(n=25\),\[ SE(\bar{X}) = \frac{3.0}{\sqrt{25}} = \frac{3.0}{5} = 0.6\text{ inches}. \]
The average height of 25 independently selected individuals varies on a scale of approximately 0.6 inches around 69.3 inches.
Sample of 100 Individuals
For \(n=100\),\[ SE(\bar{X}) = \frac{3.0}{\sqrt{100}} = \frac{3.0}{10} = 0.3\text{ inches}. \]
Sample of 1,000 Individuals
For \(n=1{,}000\),\[ SE(\bar{X}) = \frac{3.0}{\sqrt{1{,}000}} \approx 0.0949\text{ inches}. \]
The results can be summarized as follows:
| Sample size | Expected sample mean | Standard error |
|---|---|---|
| 1 | 69.3 inches | 3.000 inches |
| 25 | 69.3 inches | 0.600 inches |
| 100 | 69.3 inches | 0.300 inches |
| 1,000 | 69.3 inches | 0.095 inches |
The expected sample mean remains 69.3 inches, but its uncertainty decreases as the sample size grows.
Interpreting the Standard Error
The standard error is sometimes described informally as a “give or take” number:\[ \bar{X}\approx\mu\text{, give or take }SE(\bar{X}). \]
This is useful intuition, but one standard error is not a guaranteed bound.
If the sampling distribution of \(\bar{X}\) is approximately normal, then:\[ P\left( \mu-SE(\bar{X}) \leq \bar{X} \leq \mu+SE(\bar{X}) \right) \approx0.68, \]
and\[ P\left( \mu-2SE(\bar{X}) \leq \bar{X} \leq \mu+2SE(\bar{X}) \right) \approx0.95. \]
For \(n=100\) in the preceding example,\[ SE(\bar{X})=0.3. \]
An approximate one-standard-error range is\[ 69.3\pm0.3, \]
or\[ 69.0\text{ to }69.6\text{ inches}. \]
An approximate two-standard-error range is\[ 69.3\pm0.6, \]
or\[ 68.7\text{ to }69.9\text{ inches}. \]
These probability interpretations require an approximately normal sampling distribution. That may follow from a normal population or, for sufficiently large \(n\), from the central limit theorem.
How Sample Size Controls Precision
The square-root law can be rearranged to determine the sample size needed for a desired standard error.
Starting with\[ SE(\bar{X})=\frac{\sigma}{\sqrt{n}}, \]
suppose the desired standard error is \(s\). Then\[ s=\frac{\sigma}{\sqrt{n}}. \]
Solving for \(n\),\[ \sqrt{n}=\frac{\sigma}{s} \]
and\[ \boxed{ n=\left(\frac{\sigma}{s}\right)^2 }. \]
Example
Suppose \(\sigma=3\) inches and we want\[ SE(\bar{X})=0.1\text{ inch}. \]
Then\[ n = \left(\frac{3}{0.1}\right)^2 = 30^2 = 900. \]
A sample size of 900 is required under the independent-sampling formula.
Diminishing Returns
Because the standard error decreases according to \(\sqrt{n}\), improving precision can require a substantial increase in sample size.
To divide the standard error by 2, the sample size must be multiplied by\[ 2^2=4. \]
To divide it by 3, the sample size must be multiplied by\[ 3^2=9. \]
To divide it by 10, the sample size must be multiplied by\[ 10^2=100. \]
| Desired change in standard error | Required change in sample size |
|---|---|
| Divide SE by 2 | Multiply \(n\) by 4 |
| Divide SE by 3 | Multiply \(n\) by 9 |
| Divide SE by 5 | Multiply \(n\) by 25 |
| Divide SE by 10 | Multiply \(n\) by 100 |
This is one of the most important practical consequences of the square-root law.
Does Population Size Matter?
Under independent sampling with replacement,\[ SE(\bar{X})=\frac{\sigma}{\sqrt{n}} \]
depends on:
- Population variability \(\sigma\)
- Sample size \(n\)
It does not depend directly on the population size \(N\).
This explains why a properly selected sample of 1,000 people can estimate a characteristic of a population containing millions of people with useful precision.
The key issue is not the percentage of the population sampled. It is the absolute sample size, the variability in the population, and the quality of the sampling procedure.
Sampling Without Replacement
Most real surveys use simple random sampling without replacement. In that case, observations are not independent, and population size can affect the standard error.
The finite-population standard error is\[ SE(\bar{X}) = \frac{\sigma}{\sqrt{n}} \sqrt{\frac{N-n}{N-1}}, \]
where:
- \(N\) is the population size.
- \(n\) is the sample size.
- \(\sigma\) is the population standard deviation.
The term\[ \sqrt{\frac{N-n}{N-1}} \]
is the finite population correction.
When the Population Is Very Large
If \(N\) is much larger than \(n\), then\[ \sqrt{\frac{N-n}{N-1}}\approx1. \]
The standard error is therefore approximately\[ SE(\bar{X})\approx\frac{\sigma}{\sqrt{n}}. \]
For example, if 1,000 people are sampled from a population of 140 million,\[ \sqrt{ \frac{140{,}000{,}000-1{,}000} {140{,}000{,}000-1} } \approx0.999996. \]
The correction is negligible.
When the Sampling Fraction Is Large
If 1,000 people are sampled from a population of 2,000, then\[ \sqrt{\frac{2{,}000-1{,}000}{2{,}000-1}} \approx0.707. \]
The finite population correction now substantially reduces the standard error.
If the entire population is observed, \(n=N\), then\[ SE(\bar{X})=0. \]
There is no sampling uncertainty because a complete census has been taken.
Population size is usually negligible for a small sampling fraction, but it is not universally irrelevant.
Standard Error Does Not Measure Bias
The square-root law describes random sampling variation under the assumed sampling design. It does not account for systematic errors such as:
- Selection bias
- Nonresponse bias
- Undercoverage
- Measurement error
- Incorrect records
- A poorly defined target population
Increasing the sample size reduces standard error, but it does not necessarily reduce bias.
A large unrepresentative sample can have a very small standard error while still producing a seriously inaccurate estimate.
Population Standard Deviation Versus Estimated Standard Error
The theoretical formula uses the population standard deviation:\[ SE(\bar{X})=\frac{\sigma}{\sqrt{n}}. \]
In practice, \(\sigma\) is often unknown. It is then estimated using the sample standard deviation \(s\):\[ \widehat{SE}(\bar{X}) = \frac{s}{\sqrt{n}}. \]
The hat indicates that this is an estimated standard error.
For small samples from an approximately normal population, inference about the mean usually uses Student’s \(t\)-distribution rather than treating the estimated standard error as if \(\sigma\) were known.
Conditions Behind the Formula
The basic square-root formula is exact when observations are independent and have a common variance \(\sigma^2\).
If observations are correlated, then covariance terms must be included:\[ \operatorname{Var}(\bar{X}) = \frac{1}{n^2} \left[ \sum_{i=1}^{n}\operatorname{Var}(X_i) + 2\sum_{i<j}\operatorname{Cov}(X_i,X_j) \right]. \]
Positive correlation generally makes the standard error larger than \(\sigma/\sqrt{n}\). This is relevant for:
- Clustered survey samples
- Repeated measurements
- Time-series observations
- Spatial data
- Students sampled from the same schools
- Patients sampled from the same hospitals
In such settings, simply increasing the number of correlated observations may provide less new information than increasing the number of independent observations.
Key Takeaway
The sample mean is a random statistic whose expected value equals the population mean:\[ E(\bar{X})=\mu. \]
Its standard error is\[ SE(\bar{X}) = \frac{\sigma}{\sqrt{n}}. \]
This square-root law shows that larger independent random samples produce more precise sample averages. To cut the standard error in half, the sample size must be quadrupled.
For sampling without replacement, population size enters through the finite population correction. When the population is much larger than the sample, this correction is negligible, which is why a well-selected sample of moderate size can provide useful information about a very large population.