Constructing Confidence Intervals with Estimates, Critical Values, and Standard Errors
A confidence interval gives a range of plausible values for an unknown population parameter. Most confidence intervals combine three elements:
- A point estimate
- A critical value
- A standard error
The general structure is\[ \boxed{ \text{estimate} \pm \text{critical value}\times\text{standard error} }. \]
Population Parameters and Sample Estimates
A population parameter is a fixed numerical characteristic of a target population or probability model.
Examples include:
- Population mean \(\mu\)
- Population proportion \(p\)
- Population standard deviation \(\sigma\)
- Difference between two population means
- Regression coefficient \(\beta\)
Because observing an entire population is often impractical, the parameter is estimated using sample data.
Examples include:
| Population parameter | Sample estimate |
|---|---|
| Population mean \(\mu\) | Sample mean \(\bar{X}\) |
| Population proportion \(p\) | Sample proportion \(\hat{p}\) |
| Population standard deviation \(\sigma\) | Sample standard deviation \(s\) |
| Regression coefficient \(\beta\) | Estimated coefficient \(\hat{\beta}\) |
A confidence interval adds an uncertainty range around the point estimate.
Confidence Intervals as Plausible Ranges
Suppose \(\theta\) is an unknown parameter and \(\hat{\theta}\) is its estimator. A confidence interval takes the form\[ \hat{\theta}\pm\text{margin of error}. \]
The margin of error is usually\[ \boxed{ ME=(\text{critical value})\times SE(\hat{\theta}) }. \]
Therefore,\[ \boxed{ CI = \hat{\theta} \pm (\text{critical value})SE(\hat{\theta}) }. \]
The confidence interval expands the point estimate into a range that accounts for random sample-to-sample variation.
Example 1: A Population Approval Proportion
Suppose the parameter of interest is\[ p = \text{proportion of likely voters who approve}. \]
A random sample contains \(n\) voters. Define an indicator variable:\[ X_i= \begin{cases} 1, & \text{if voter }i\text{ approves},\\ 0, & \text{otherwise}. \end{cases} \]
The sample proportion is\[ \hat{p} = \frac{X_1+\cdots+X_n}{n}. \]
Because \(\hat{p}\) is an average of 0–1 indicators, its expected value is\[ E(\hat{p})=p. \]
Under independent sampling,\[ SE(\hat{p}) = \sqrt{ \frac{p(1-p)}{n} }. \]
Since \(p\) is unknown, the standard error is estimated using\[ \boxed{ \widehat{SE}(\hat{p}) = \sqrt{ \frac{\hat{p}(1-\hat{p})}{n} } }. \]
An approximate confidence interval is then\[ \boxed{ \hat{p} \pm z^* \sqrt{ \frac{\hat{p}(1-\hat{p})}{n} } }. \]
Example 2: Repeated Measurements
Suppose a physical quantity \(\mu\) is measured 30 times:\[ X_1,X_2,\ldots,X_{30}. \]
A natural estimate is the sample mean:\[ \bar{X} = \frac{X_1+\cdots+X_{30}}{30}. \]
If the measurements follow the model\[ X_i=\mu+\varepsilon_i \]
with errors centered at zero,\[ E(\varepsilon_i)=0, \]
then\[ E(\bar{X})=\mu. \]
If the measurement errors are independent with standard deviation \(\sigma\),\[ SE(\bar{X}) = \frac{\sigma}{\sqrt{30}}. \]
Because \(\sigma\) is normally unknown, it is estimated with the sample standard deviation \(s\):\[ \widehat{SE}(\bar{X}) = \frac{s}{\sqrt{30}}. \]
For a mean with unknown \(\sigma\), the standard confidence interval is\[ \boxed{ \bar{X} \pm t^*_{n-1} \frac{s}{\sqrt{n}} }. \]
For \(n=30\), the degrees of freedom are\[ df=29. \]
The Role of the Central Limit Theorem
Suppose \(X_1,\ldots,X_n\) are independent and identically distributed with mean \(\mu\) and finite variance \(\sigma^2\).
The central limit theorem states that\[ \frac{\bar{X}-\mu}{\sigma/\sqrt{n}} \]
approaches a standard normal distribution as \(n\) grows.
Therefore, for sufficiently large \(n\),\[ \bar{X} \approx N\left( \mu,\frac{\sigma^2}{n} \right). \]
This normal approximation motivates intervals of the form\[ \bar{X} \pm z^* \frac{\sigma}{\sqrt{n}}. \]
However, the fact that a statistic is an average does not guarantee that the approximation is accurate for every sample size. The quality of the approximation depends on:
- Sample size
- Skewness
- Tail behavior
- Outliers
- Independence
- Existence of a finite variance
If the population is approximately normal, a \(t\)-interval for the mean can work well even with a relatively small sample. If the distribution is strongly skewed or heavy-tailed, 30 observations may or may not be sufficient.
Choosing a Critical Value
For a two-sided confidence interval based on the standard normal distribution, the critical value \(z^*\) is chosen so that the desired confidence level lies between \(-z^*\) and \(z^*\):\[ P(-z^*\leq Z\leq z^*) = \text{confidence level}. \]
Common values are:
| Confidence level | \(z^*\) |
|---|---|
| 80% | 1.282 |
| 90% | 1.645 |
| 95% | 1.960 |
| 98% | 2.326 |
| 99% | 2.576 |
Rounded values are sometimes used:
- 90%: \(z^*\approx1.65\)
- 95%: \(z^*\approx2\)
- 99%: \(z^*\approx2.58\)
For accurate calculations, the unrounded critical value is preferable.
How Critical Values Are Obtained
For a confidence level \(C\), the total probability outside the interval is\[ \alpha=1-C. \]
For a two-sided interval, this is divided equally between the two tails:\[ \frac{\alpha}{2}. \]
The critical value is therefore\[ \boxed{ z^* = \Phi^{-1} \left( 1-\frac{\alpha}{2} \right) }, \]
where \(\Phi^{-1}\) is the inverse standard normal cumulative distribution function.
For a 95% interval,\[ C=0.95, \qquad \alpha=0.05. \]
Thus,\[ z^* = \Phi^{-1}(0.975) \approx 1.96. \]
Why Higher Confidence Produces Wider Intervals
For fixed data and standard error,\[ CI = \hat{\theta} \pm z^*SE(\hat{\theta}). \]
A higher confidence level requires a larger \(z^*\). Therefore, it produces a larger margin of error.
For the same estimate and standard error:\[ 99\%\text{ interval} \]
is wider than\[ 95\%\text{ interval}, \]
which is wider than\[ 90\%\text{ interval}. \]
This represents a tradeoff:
- Higher confidence gives greater long-run coverage.
- Lower confidence gives a narrower interval.
The Square-Root Law
For an average,\[ SE(\bar{X}) = \frac{\sigma}{\sqrt{n}}. \]
For a proportion,\[ SE(\hat{p}) = \frac{\sqrt{p(1-p)}}{\sqrt{n}}. \]
The standard error decreases at the rate\[ \frac{1}{\sqrt{n}}. \]
This means:
- Four times the sample size produces half the standard error.
- Nine times the sample size produces one-third the standard error.
- One hundred times the sample size produces one-tenth the standard error.
Larger samples improve precision, but with diminishing returns.
The Unknown Population Standard Deviation
The theoretical interval for a mean would use\[ \bar{X} \pm z^* \frac{\sigma}{\sqrt{n}}. \]
In practice, \(\sigma\) is rarely known. We estimate it with \(s\):\[ \sigma\longrightarrow s. \]
This produces the estimated standard error\[ \boxed{ \widehat{SE}(\bar{X}) = \frac{s}{\sqrt{n}} }. \]
However, estimating \(\sigma\) introduces additional uncertainty. For inference about a mean, this is handled with Student’s \(t\)-distribution:\[ \boxed{ \bar{X} \pm t^*_{n-1} \frac{s}{\sqrt{n}} }. \]
Normal and Student’s \(t\) Critical Values
The \(t\)-distribution resembles the standard normal distribution but has heavier tails. Its shape depends on the degrees of freedom.
For a 95% interval:
| Degrees of freedom | \(t^*\) |
|---|---|
| 5 | 2.571 |
| 10 | 2.228 |
| 20 | 2.086 |
| 29 | 2.045 |
| 100 | 1.984 |
| Infinite | 1.960 |
As the sample size grows,\[ t^*_{n-1}\longrightarrow1.96. \]
Thus, \(z\)- and \(t\)-intervals become nearly identical for large samples.
The Plug-In Principle
Replacing an unknown population quantity with its sample estimate is known as the plug-in principle.
Examples include:\[ p\longrightarrow\hat{p} \]
and\[ \sigma\longrightarrow s. \]
For a proportion,\[ \sqrt{ \frac{p(1-p)}{n} } \]
becomes\[ \sqrt{ \frac{\hat{p}(1-\hat{p})}{n} }. \]
For a mean,\[ \frac{\sigma}{\sqrt{n}} \]
becomes\[ \frac{s}{\sqrt{n}}. \]
The plug-in principle is one of the most common strategies in statistical inference.
How the Plug-In Principle Relates to the Bootstrap
The term bootstrap principle is sometimes used informally for the idea that the sample can stand in for the unknown population.
Modern computational bootstrapping is a more specific method:
- Treat the observed sample as an empirical population.
- Draw samples with replacement from it.
- Recalculate the statistic for each resample.
- Use the distribution of the resampled statistics to estimate uncertainty.
Simply replacing \(\sigma\) with \(s\) is more precisely described as plug-in estimation. It does not require resampling.
Confidence Interval for a Proportion: Worked Example
Suppose:\[ n=1{,}000 \]
and\[ \hat{p}=0.58. \]
The estimated standard error is\[ \widehat{SE}(\hat{p}) = \sqrt{ \frac{0.58(0.42)}{1{,}000} } \approx 0.01561. \]
For a 95% interval,\[ z^*=1.96. \]
The margin of error is\[ ME = 1.96(0.01561) \approx 0.0306. \]
Therefore,\[ CI = 0.58\pm0.0306, \]
giving\[ \boxed{ (0.549,\ 0.611) }. \]
On the percentage scale:\[ \boxed{ 54.9\%\text{ to }61.1\% }. \]
Confidence Interval for a Mean: Worked Structure
Suppose 30 repeated measurements produce:\[ \bar{x}=m \]
and\[ s=q. \]
The estimated standard error is\[ \widehat{SE}(\bar{X}) = \frac{q}{\sqrt{30}}. \]
Using a 95% \(t\)-interval with 29 degrees of freedom,\[ t^*_{29}\approx2.045. \]
The interval is\[ \boxed{ m \pm 2.045 \frac{q}{\sqrt{30}} }. \]
The actual endpoints require the observed values of \(m\) and \(q\).
Conditions for a Proportion Interval
The simple normal interval for a proportion requires an adequate number of observed successes and failures.
A common sample-based guideline is\[ n\hat{p}\geq10 \]
and\[ n(1-\hat{p})\geq10. \]
For \(\hat{p}=0.58\) and \(n=1{,}000\),\[ n\hat{p}=580 \]
and\[ n(1-\hat{p})=420. \]
The guideline is easily satisfied.
For smaller samples or proportions near 0 or 1, the Wilson score interval generally performs better than the simple Wald interval.
Conditions for a Mean Interval
A conventional one-sample \(t\)-interval assumes:
- Observations are independent or appropriately modeled.
- The sample represents the target population or process.
- The population is approximately normal, or the sample is large enough for the mean’s sampling distribution to be approximately normal.
- Extreme outliers do not dominate the sample.
The \(t\)-procedure is robust to moderate nonnormality, especially with larger samples, but no fixed sample size guarantees adequacy for every population distribution.
Confidence Intervals Do Not Correct Bias
The formula\[ \text{estimate} \pm \text{critical value}\times SE \]
accounts for sampling variability under the assumed model. It does not automatically account for:
- Selection bias
- Nonresponse bias
- Measurement bias
- Instrument miscalibration
- Confounding
- Dependence
- Data-processing errors
- Model misspecification
A narrow confidence interval can still be centered on a biased estimate.
Interpretation of a Confidence Interval
Under the frequentist interpretation, a 95% confidence procedure is designed so that approximately 95% of intervals constructed in repeated samples contain the true parameter.
After one interval has been calculated, the parameter is fixed and the interval either covers it or does not.
A practical statement is:
We are 95% confident that the parameter lies between the reported endpoints.
This refers to confidence in the long-run reliability of the procedure under its assumptions.
Key Takeaway
Most confidence intervals combine three elements:\[ \boxed{ \text{estimate} \pm \text{critical value}\times\text{standard error} }. \]
For a proportion,\[ \hat{p} \pm z^* \sqrt{ \frac{\hat{p}(1-\hat{p})}{n} } \]
provides a common large-sample interval.
For a mean with unknown population standard deviation,\[ \bar{X} \pm t^*_{n-1} \frac{s}{\sqrt{n}} \]
is generally appropriate.
The critical value is determined by the confidence level, while the standard error measures sample-to-sample variability. Replacing unknown population quantities with sample estimates is the plug-in principle. Computational bootstrapping extends this idea through repeated resampling when analytic standard-error formulas are unavailable or inconvenient.