Estimating Unknown Standard Errors with the Plug-In and Bootstrap Principles
Confidence intervals require a standard error, but standard-error formulas often contain unknown population quantities. A practical solution is to replace the unknown quantities with estimates calculated from the observed sample.
This idea is commonly called the plug-in principle. It is closely related to the bootstrap principle, although it should be distinguished from computational bootstrap resampling.
The General Problem
A confidence interval often has the form\[ \text{estimate} \pm z^*\times SE(\text{estimate}). \]
For a population proportion \(p\), the sample proportion \(\hat{p}\) has standard error\[ SE(\hat{p}) = \sqrt{ \frac{p(1-p)}{n} }. \]
The difficulty is that \(p\) is unknown. If we already knew \(p\), there would be little reason to estimate it with a confidence interval.
We therefore need to estimate the standard error from the sample.
The Plug-In Principle
The plug-in principle says:
Replace an unknown population quantity with its sample estimate.
For a population proportion,\[ p \]
is replaced with\[ \hat{p}. \]
Therefore,\[ SE(\hat{p}) = \sqrt{ \frac{p(1-p)}{n} } \]
is estimated by\[ \boxed{ \widehat{SE}(\hat{p}) = \sqrt{ \frac{\hat{p}(1-\hat{p})}{n} } }. \]
This estimated standard error can then be used to construct a confidence interval.
Example: Presidential Approval Rating
Suppose a poll samples\[ n=1{,}000 \]
likely voters, and 58% approve of the president’s job performance.
The sample proportion is\[ \hat{p}=0.58. \]
The true population proportion \(p\) is unknown.
Using the plug-in principle,\[ \widehat{SE}(\hat{p}) = \sqrt{ \frac{0.58(1-0.58)}{1{,}000} }. \]
Since\[ 1-0.58=0.42, \]
we obtain\[ \widehat{SE}(\hat{p}) = \sqrt{ \frac{0.58(0.42)}{1{,}000} }. \]
Calculate the product:\[ 0.58(0.42)=0.2436. \]
Thus,\[ \widehat{SE}(\hat{p}) = \sqrt{0.0002436} \approx 0.01561. \]
On the percentage scale, this is\[ 0.01561(100) \approx 1.56 \]
percentage points.
Constructing the 95% Confidence Interval
Using the standard normal critical value\[ z^*=1.96, \]
the margin of error is\[ ME = 1.96(0.01561) \approx 0.03059. \]
Therefore,\[ \hat{p}\pm ME = 0.58\pm0.03059. \]
The endpoints are\[ 0.58-0.03059 = 0.54941 \]
and\[ 0.58+0.03059 = 0.61059. \]
Thus, the approximate 95% confidence interval is\[ \boxed{ (0.549,\ 0.611) }. \]
Expressed as percentages:\[ \boxed{ 54.9\%\text{ to }61.1\% }. \]
If the critical value is rounded to 2, the result remains approximately the same:\[ 0.58 \pm 2 \sqrt{ \frac{0.58(0.42)}{1{,}000} }. \]
Why the Estimated Bernoulli Standard Deviation Is About 0.49
For a Bernoulli random variable,\[ X= \begin{cases} 1, & \text{success},\\ 0, & \text{failure}, \end{cases} \]
the population variance is\[ \operatorname{Var}(X)=p(1-p). \]
The population standard deviation is\[ \sigma=\sqrt{p(1-p)}. \]
Replacing \(p\) with \(\hat{p}=0.58\) gives\[ \hat{\sigma} = \sqrt{0.58(0.42)} = \sqrt{0.2436} \approx 0.4936. \]
Rounded to two decimal places,\[ \boxed{\hat{\sigma}\approx0.49}. \]
The estimated standard error is then\[ \widehat{SE}(\hat{p}) = \frac{\hat{\sigma}}{\sqrt{n}} = \frac{0.4936}{\sqrt{1{,}000}} \approx 0.01561. \]
A Small Denominator Detail
The quantity\[ \sqrt{\hat{p}(1-\hat{p})} \]
is the empirical standard deviation when the squared deviations are averaged using denominator \(n\).
The conventional sample standard deviation uses denominator \(n-1\):\[ s = \sqrt{ \frac{1}{n-1} \sum_{i=1}^{n}(X_i-\bar{X})^2 }. \]
For 0–1 data, this becomes\[ s = \sqrt{ \frac{n}{n-1} \hat{p}(1-\hat{p}) }. \]
With \(n=1{,}000\), the difference between \(n\) and \(n-1\) is negligible. However, it is useful to distinguish the exact formulas.
Plug-In Principle Versus Bootstrap Resampling
The term bootstrap principle is sometimes used broadly to mean that the sample is treated as an approximation to the population. Under that broad idea, population quantities are replaced by corresponding sample quantities.
In modern statistical practice, however, the bootstrap usually refers to a specific computational procedure:
- Treat the observed sample as an empirical population.
- Draw a new sample of size \(n\) from it with replacement.
- Recalculate the statistic.
- Repeat this process many times.
- Use the variability of the bootstrap statistics to estimate the standard error.
Simply replacing \(p\) with \(\hat{p}\) in a known formula is more precisely called a plug-in estimate. It does not require repeated resampling.
Computational Bootstrap for the Approval Rating
Suppose the observed sample contains:
- 580 approval labels equal to 1
- 420 nonapproval labels equal to 0
A bootstrap procedure would repeatedly draw 1,000 labels with replacement from these observed labels.
For bootstrap sample \(b\), calculate\[ \hat{p}^{*(b)}. \]
After \(B\) bootstrap samples, estimate the standard error with\[ \boxed{ SE_{\text{boot}} = SD\left( \hat{p}^{*(1)}, \hat{p}^{*(2)}, \ldots, \hat{p}^{*(B)} \right) }. \]
For a simple sample proportion, this bootstrap standard error will be close to\[ \sqrt{ \frac{\hat{p}(1-\hat{p})}{n} }. \]
The formula is preferable in this simple setting because it is exact under the plug-in Bernoulli model and does not involve Monte Carlo variation. Bootstrap resampling becomes especially valuable when no convenient standard-error formula is available.
A Measurement Example
Suppose the speed of light is measured 30 times:\[ Y_1,Y_2,\ldots,Y_{30}. \]
A simple measurement model is\[ Y_i=\theta+\varepsilon_i, \]
where:
- \(\theta\) is the true physical quantity.
- \(\varepsilon_i\) is the measurement error for observation \(i\).
To interpret the sample mean as estimating \(\theta\), we usually assume\[ E(\varepsilon_i)=0. \]
Then\[ E(Y_i) = \theta+E(\varepsilon_i) = \theta. \]
The sample mean\[ \bar{Y} = \frac{1}{30} \sum_{i=1}^{30}Y_i \]
is therefore an unbiased estimator of \(\theta\).
Why the Measurement and Error Standard Deviations Match
Because\[ Y_i=\theta+\varepsilon_i \]
and \(\theta\) is fixed,\[ \operatorname{Var}(Y_i) = \operatorname{Var}(\theta+\varepsilon_i) = \operatorname{Var}(\varepsilon_i). \]
Adding a constant does not change variance:\[ \operatorname{Var}(X+c)=\operatorname{Var}(X). \]
Therefore,\[ SD(Y_i)=SD(\varepsilon_i). \]
This means the standard deviation of the observed measurements can estimate the standard deviation of the random measurement errors.
If the observed measurements have sample standard deviation \(s\), then\[ s \]
is used to estimate the unknown error standard deviation\[ \sigma. \]
Standard Error of the Mean Measurement
If the measurement errors are independent and have standard deviation \(\sigma\), then\[ SE(\bar{Y}) = \frac{\sigma}{\sqrt{n}}. \]
Because \(\sigma\) is unknown, estimate it using \(s\):\[ \boxed{ \widehat{SE}(\bar{Y}) = \frac{s}{\sqrt{n}} }. \]
For \(n=30\),\[ \widehat{SE}(\bar{Y}) = \frac{s}{\sqrt{30}}. \]
This is another application of the plug-in principle.
Confidence Interval for a Mean with Unknown Variance
When the population standard deviation is unknown, a confidence interval for a mean generally uses Student’s \(t\)-distribution:\[ \boxed{ \bar{Y} \pm t^*_{n-1} \frac{s}{\sqrt{n}} }. \]
For \(n=30\), the degrees of freedom are\[ df=29. \]
For a 95% interval,\[ t^*_{29}\approx2.045. \]
Therefore, the interval is\[ \boxed{ \bar{Y} \pm 2.045 \frac{s}{\sqrt{30}} }. \]
Using \(z^*=1.96\) would give a similar result, but the \(t\)-interval accounts for the additional uncertainty introduced by estimating \(\sigma\).
Example with Hypothetical Measurements
Suppose 30 measurements have:\[ \bar{Y}=299{,}800 \]
and\[ s=110 \]
in the chosen measurement units.
The estimated standard error is\[ \widehat{SE}(\bar{Y}) = \frac{110}{\sqrt{30}} \approx 20.08. \]
The 95% margin of error is\[ ME = 2.045(20.08) \approx 41.06. \]
The confidence interval is therefore\[ 299{,}800\pm41.06, \]
or approximately\[ \boxed{ (299{,}758.9,\ 299{,}841.1) }. \]
This interval represents random measurement uncertainty under the assumed model.
Computational Bootstrap for the Mean
Instead of using the \(t\)-formula, the standard error can be estimated through bootstrap resampling:
- Begin with the 30 observed measurements.
- Draw 30 values with replacement.
- Calculate the resampled mean \(\bar{Y}^*\).
- Repeat many times.
- Calculate the standard deviation of the resampled means.
The bootstrap standard error is\[ SE_{\text{boot}} = SD(\bar{Y}^{*(1)},\ldots,\bar{Y}^{*(B)}). \]
For an ordinary sample mean, this result will usually be close to\[ \frac{s}{\sqrt{n}}. \]
The classical \(t\)-interval is generally preferable when its assumptions are appropriate. The computational bootstrap is especially useful for statistics whose sampling distributions or standard errors are difficult to derive.
Bootstrap Confidence-Interval Methods
Bootstrap resampling can produce confidence intervals in several ways.
Normal Bootstrap Interval
\[ \hat{\theta} \pm z^*SE_{\text{boot}}. \]
This assumes the estimator’s sampling distribution is approximately symmetric and normal.
Percentile Bootstrap Interval
If the bootstrap estimates are sorted, the central 95% can be used directly. For example, the 2.5th and 97.5th percentiles form a percentile interval.
Basic Bootstrap Interval
If \(q_{0.025}^*\) and \(q_{0.975}^*\) are bootstrap quantiles, the basic interval is\[ \left( 2\hat{\theta}-q_{0.975}^*, \, 2\hat{\theta}-q_{0.025}^* \right). \]
Bias-Corrected and Accelerated Interval
The BCa interval adjusts for bias and asymmetry and is often more accurate, though more computationally involved.
Different bootstrap intervals have different properties. The phrase “a bootstrap confidence interval” does not identify a unique procedure.
What the Bootstrap Assumes
The bootstrap treats the observed sample as a useful approximation to the population distribution. It works best when:
- The observations are representative.
- The sampling units are independent or dependence is resampled appropriately.
- The sample is large enough to represent important distributional features.
- The statistic is reasonably stable.
- The resampling scheme matches the original study design.
A small sample may fail to contain important rare outcomes or tail behavior. Resampling cannot create population features absent from the observed data.
Dependent and Complex Data
Ordinary resampling of individual observations is inappropriate when observations are dependent.
Examples include:
- Time-series data
- Clustered survey samples
- Repeated measurements
- Spatial data
- Matched pairs
Possible alternatives include:
- Block bootstrap
- Cluster bootstrap
- Paired bootstrap
- Stratified bootstrap
- Survey-design bootstrap methods
The resampling unit must reflect the structure that generated the original data.
Random Error Versus Systematic Error
The measurement model\[ Y_i=\theta+\varepsilon_i \]
with\[ E(\varepsilon_i)=0 \]
assumes errors are centered at zero.
If instead\[ E(\varepsilon_i)=b\neq0, \]
then\[ E(Y_i)=\theta+b. \]
The sample mean estimates \(\theta+b\), not \(\theta\). Repeated measurements can reduce random error but do not eliminate systematic bias.
Neither the plug-in principle nor ordinary bootstrap resampling automatically detects or corrects:
- Instrument miscalibration
- Consistent timing errors
- Selection bias
- Nonresponse bias
- Incorrect model assumptions
- Measurement drift
- Data-processing mistakes
Confidence Intervals for Polls Also Require Design Validity
The approval-rating interval assumes something close to independent representative sampling. Real polls may involve:
- Sampling weights
- Stratification
- Clustering
- Nonresponse adjustments
- Likely-voter models
- Coverage limitations
A naive plug-in interval may be too narrow if these features increase variance. Survey-specific standard errors should account for the actual design.
Key Takeaway
When a standard-error formula contains an unknown population quantity, the plug-in principle replaces it with a sample estimate.
For a population proportion,\[ SE(\hat{p}) = \sqrt{ \frac{p(1-p)}{n} } \]
is estimated by\[ \boxed{ \widehat{SE}(\hat{p}) = \sqrt{ \frac{\hat{p}(1-\hat{p})}{n} } }. \]
For \(\hat{p}=0.58\) and \(n=1{,}000\), the approximate 95% confidence interval is\[ \boxed{ 54.9\%\text{ to }61.1\% }. \]
For a sample mean with unknown population standard deviation,\[ SE(\bar{Y})=\frac{\sigma}{\sqrt{n}} \]
is estimated by\[ \boxed{ \widehat{SE}(\bar{Y}) = \frac{s}{\sqrt{n}} }. \]
Simply substituting sample quantities into a known formula is most precisely called the plug-in principle. Computational bootstrap resampling extends the same general idea by repeatedly sampling with replacement from the empirical distribution, making it possible to estimate uncertainty for more complicated statistics.