Paired t-Tests and Sign Tests for Matched Data
When observations naturally occur in pairs, the analysis should preserve that pairing. Examples include:
- Husbands and wives within couples
- Measurements before and after treatment
- Twins or matched participants
- Two methods applied to the same specimen
- Repeated measurements of the same person
Treating paired observations as two independent samples discards important information and can produce an inappropriate standard error.
A Paired Age Example
Suppose we observe the ages of five heterosexual married couples:
| Couple | Husband’s age | Wife’s age |
|---|---|---|
| 1 | 43 | 41 |
| 2 | 71 | 70 |
| 3 | 32 | 31 |
| 4 | 68 | 66 |
| 5 | 27 | 26 |
Every husband in this small sample is older than his wife. The question is whether these data provide evidence of a corresponding population pattern.
The target population must be defined carefully—for example, currently married heterosexual couples in a specified place and period. The results cannot automatically be generalized beyond the population represented by the sampling process.
Why the Two Samples Are Dependent
The husbands’ ages are\[ (43,71,32,68,27), \]
and the wives’ ages are\[ (41,70,31,66,26). \]
These should not be analyzed as two independent samples. Each husband is specifically matched with one wife, and ages within a couple tend to be similar.
For example:
- The 71-year-old husband is paired with the 70-year-old wife.
- The 27-year-old husband is paired with the 26-year-old wife.
The observations within each pair are related.
An independent two-sample procedure would ignore this relationship and compare the large between-person age variation—people in their 20s through 70s—even though the question concerns the much smaller within-couple differences.
Convert Paired Data into Differences
Define the paired difference as\[ D_i = \text{husband’s age} – \text{wife’s age}. \]
The observed differences are\[ (2,1,1,2,1). \]
Once these differences are calculated, the problem becomes a one-sample analysis.
| Couple | Difference \(D_i\) |
|---|---|
| 1 | 2 |
| 2 | 1 |
| 3 | 1 |
| 4 | 2 |
| 5 | 1 |
A positive difference means the husband is older. A negative difference would mean the wife is older.
Why Pairing Improves Precision
Suppose individual ages vary widely, but the ages within each couple are close. The paired analysis removes much of the irrelevant between-couple variation.
For paired variables \(H\) and \(W\),\[ D=H-W. \]
The variance of the difference is\[ \operatorname{Var}(D) = \operatorname{Var}(H) + \operatorname{Var}(W) – 2\operatorname{Cov}(H,W). \]
If husband and wife ages are positively correlated, then\[ \operatorname{Cov}(H,W)>0. \]
The covariance term reduces the variance of the difference. That reduction can make the paired analysis substantially more precise than an independent-samples analysis.
The Paired t-Test
The paired t-test is a one-sample t-test applied to the differences.
Let\[ \mu_D=E(D) \]
be the population mean paired difference.
For a two-sided test:\[ H_0:\mu_D=0 \]
versus\[ H_A:\mu_D\neq0. \]
If the research question was specified in advance as whether husbands are older on average, a one-sided alternative is\[ H_A:\mu_D>0. \]
The direction must be selected before examining the data.
Mean Difference
The observed differences are\[ 2,1,1,2,1. \]
Their mean is\[ \bar{D} = \frac{2+1+1+2+1}{5} = \frac{7}{5} = 1.4. \]
Thus, the husbands in this sample are, on average,\[ \boxed{1.4\text{ years older}}. \]
Standard Deviation of the Differences
The deviations from the mean are\[ 0.6,-0.4,-0.4,0.6,-0.4. \]
Their squared values are\[ 0.36,0.16,0.16,0.36,0.16. \]
The sum of squared deviations is\[ 0.36+0.16+0.16+0.36+0.16=1.20. \]
The sample variance is\[ s_D^2 = \frac{1.20}{5-1} = 0.30. \]
Therefore,\[ s_D = \sqrt{0.30} \approx 0.548. \]
Rounded,\[ \boxed{s_D\approx0.55\text{ years}}. \]
Standard Error of the Mean Difference
The estimated standard error is\[ \widehat{SE}(\bar{D}) = \frac{s_D}{\sqrt{n}}. \]
With \(n=5\),\[ \widehat{SE}(\bar{D}) = \frac{0.548}{\sqrt{5}} \approx 0.245. \]
Thus,\[ \boxed{ \widehat{SE}(\bar{D}) \approx0.245\text{ years} }. \]
Calculating the t-Statistic
The paired t-statistic is\[ t = \frac{\bar{D}-\mu_{D,0}} {s_D/\sqrt{n}}, \]
where \(\mu_{D,0}\) is the mean difference under \(H_0\).
Since\[ \mu_{D,0}=0, \]
we obtain\[ t = \frac{1.4-0}{0.548/\sqrt{5}}. \]
Therefore,\[ t \approx \frac{1.4}{0.245} \approx 5.72. \]
The small difference between 5.69 and 5.72 is due to rounding.
The degrees of freedom are\[ df=n-1=4. \]
P-Value for the Paired t-Test
For the two-sided alternative,\[ H_A:\mu_D\neq0, \]
the p-value is\[ P(|T_4|\geq5.72). \]
This is approximately\[ \boxed{p\approx0.0046}. \]
That is about 0.46%.
For the preselected one-sided alternative\[ H_A:\mu_D>0, \]
the p-value is approximately half as large:\[ \boxed{p\approx0.0023}. \]
Under the paired t-test model, the sample provides strong evidence against a population mean difference of zero.
Confidence Interval for the Mean Difference
A 95% confidence interval is\[ \bar{D} \pm t^*_{4} \frac{s_D}{\sqrt{n}}. \]
For 4 degrees of freedom,\[ t^*_{4}\approx2.776. \]
Therefore,\[ 1.4 \pm 2.776(0.245). \]
The margin of error is approximately\[ 0.680. \]
Thus, the interval is\[ \boxed{ (0.72,\ 2.08)\text{ years} }. \]
Under the paired t-test assumptions, the estimated population mean age difference lies between approximately 0.72 and 2.08 years.
The interval is more informative than the p-value alone because it describes the plausible magnitude of the mean difference.
Assumptions of the Paired t-Test
The paired t-test does not require the two measurements within a pair to be independent. Their dependence is the reason for using the paired method.
It does require that:
- The pairs are sampled independently of one another.
- The pairing is meaningful and correctly recorded.
- The differences are quantitative.
- The population distribution of differences is approximately normal, particularly for very small samples.
- Extreme outlying differences do not dominate the result.
- The sample represents the target population appropriately.
The relevant normality assumption concerns\[ D_i=H_i-W_i, \]
not the separate distributions of husbands’ and wives’ ages.
Caution with Only Five Pairs
The calculated t-statistic is large because the five observed differences are highly consistent:\[ (2,1,1,2,1). \]
However, \(n=5\) is extremely small. With so few pairs:
- Normality of the population differences is difficult to assess.
- One unusual pair could change the result substantially.
- Population representativeness is critical.
- The reported precision depends heavily on the model assumptions.
A small p-value does not compensate for a convenience sample or an ill-defined target population.
What If Only the Direction Is Known?
Suppose we do not know the numerical age difference for each couple. We know only whether:
- The husband is older
- The wife is older
Define\[ S_i= \begin{cases} 1, & D_i>0,\\ 0, & D_i<0. \end{cases} \]
The sample contains five positive differences:\[ (1,1,1,1,1). \]
The sign test evaluates the probability of a positive difference.
Hypotheses for the Sign Test
Let\[ \pi=P(D>0). \]
For a continuous difference distribution with no ties, the null hypothesis is\[ H_0:\pi=0.5. \]
A two-sided alternative is\[ H_A:\pi\neq0.5. \]
A directional alternative is\[ H_A:\pi>0.5. \]
This is not the same parameter tested by the paired t-test:
- Paired t-test: population mean difference \(\mu_D\)
- Sign test: probability of a positive difference \(\pi\), often interpreted through the median under suitable continuity conditions
Exact Sign-Test Distribution
Under\[ H_0:\pi=0.5, \]
the number of positive differences \(S\) follows\[ S\sim\operatorname{Binomial}(5,0.5). \]
The observed count is\[ S_{\text{obs}}=5. \]
For the one-sided alternative,\[ H_A:\pi>0.5, \]
the exact p-value is\[ P(S\geq5) = P(S=5). \]
Therefore,\[ P(S=5) = \binom{5}{5}(0.5)^5 = \frac{1}{32} = 0.03125. \]
Thus,\[ \boxed{ p_{\text{one-sided}}=0.03125 }. \]
For a conventional equal-tail two-sided exact test,\[ p_{\text{two-sided}} = 2\left(\frac{1}{32}\right) = \frac{1}{16} = 0.0625. \]
Therefore,\[ \boxed{ p_{\text{two-sided}}=0.0625 }. \]
At the 5% significance level:
- The one-sided exact sign test rejects \(H_0\).
- The two-sided exact sign test does not reject \(H_0\).
Why the Normal Approximation Is Misleading Here
A normal approximation to the sign count uses\[ E(S)=n(0.5)=2.5 \]
and\[ SE(S) = \sqrt{n(0.5)(0.5)} = \sqrt{1.25} \approx1.118. \]
Without a continuity correction,\[ z = \frac{5-2.5}{1.118} \approx 2.24. \]
This gives a one-sided normal-tail probability near 1.25% and a two-sided value near 2.5%.
But this approximation is poor because\[ n=5 \]
is far too small for an accurate normal approximation to the binomial distribution.
The exact p-values—3.125% one-sided and 6.25% two-sided—should be used.
For five pairs, the claim that the two-sided sign test is significant based on \(z=2.24\) is incorrect. The exact two-sided p-value is 0.0625.
A continuity correction improves the normal approximation but still does not justify replacing the simple exact calculation.
How Ties Are Handled
If\[ D_i=0, \]
neither member of the pair is larger. The traditional sign test usually removes tied pairs and performs the binomial test using only nonzero differences.
If there are:
- \(n_+\) positive differences
- \(n_-\) negative differences
- \(n_0\) ties
then the effective sample size is\[ n_{\text{eff}}=n_++n_-. \]
The number of positive signs is compared with\[ \operatorname{Binomial}(n_{\text{eff}},0.5). \]
A large number of ties may itself contain important information and should be reported.
Why the Sign Test Is Less Powerful
The paired t-test uses the complete differences:\[ (2,1,1,2,1). \]
The sign test reduces them to:\[ (+,+,+,+,+). \]
It discards whether a difference is:
- 0.1 years
- 1 year
- 10 years
All positive values count identically.
This loss of magnitude information generally reduces statistical power when the paired t-test assumptions are reasonable.
In the example:
- Paired t-test, two-sided: \(p\approx0.0046\)
- Exact sign test, two-sided: \(p=0.0625\)
The difference is substantial because the t-test uses the small and consistent magnitudes, whereas the sign test uses only five positive signs.
Robustness of the Sign Test
The sign test requires fewer distributional assumptions than the paired t-test.
It does not require normally distributed differences and is resistant to extreme magnitudes because it uses only signs.
Its advantages include:
- Simple interpretation
- Applicability to ordinal paired comparisons
- Resistance to outliers
- Exact small-sample calculations
Its disadvantages include:
- Loss of magnitude information
- Lower power in many settings
- Limited description of effect size
- Potential complications from ties
The Wilcoxon Signed-Rank Test
A middle-ground procedure is the Wilcoxon signed-rank test.
It uses:
- The direction of each difference
- The rank of its absolute magnitude
It retains more information than the sign test but generally requires stronger assumptions, including an approximately symmetric distribution of paired differences for its usual location interpretation.
The three common paired procedures can be compared as follows:
| Procedure | Information used | Main target |
|---|---|---|
| Paired t-test | Numerical differences | Mean difference |
| Wilcoxon signed-rank | Signs and magnitude ranks | Location shift under symmetry |
| Sign test | Signs only | Probability positive or median-related location |
Independent and Paired Data Must Not Be Confused
Independent Samples
Examples include:
- Different patients in treatment and control groups
- Students from separate schools
- Customers assigned to different website designs
An independent two-sample method is appropriate when observations in one group are unrelated to observations in the other.
Paired Samples
Examples include:
- Before and after measurements
- Matched cases and controls
- Two measurements from the same person
- Spouses within couples
A paired method analyzes within-pair differences.
Using an independent-samples test for paired data usually wastes information and may produce an inappropriate standard error. Using a paired test for genuinely independent data invents a pairing that has no scientific meaning.
Study Design Still Matters
Neither a paired t-test nor a sign test creates a representative sample.
If the five couples were selected because they were convenient, the results cannot automatically be generalized to a broad population. Potential issues include:
- Age-cohort effects
- Marriage patterns
- Geographic differences
- Selection into marriage
- Divorce and remarriage
- Different definitions of the target population
- Changes across historical periods
Statistical significance quantifies sampling variation under a model. It does not correct selection bias.
Practical Workflow for Paired Data
- Confirm that the observations are genuinely paired.
- Define the direction of the difference consistently.
- Plot or list the paired differences.
- Identify ties and unusual values.
- Define the population parameter.
- Choose a one-sided or two-sided alternative in advance.
- Use a paired t-test when mean differences and its assumptions are relevant.
- Use an exact sign test when only direction is meaningful or robust inference is needed.
- Report an effect estimate and confidence interval.
- State the target population and sampling limitations.
Key Takeaway
Paired observations should be analyzed through their within-pair differences:\[ D_i=X_i-Y_i. \]
The paired t-statistic is\[ t = \frac{\bar{D}-\mu_{D,0}} {s_D/\sqrt{n}}. \]
For differences\[ (2,1,1,2,1), \]
we obtain\[ \bar{D}=1.4, \qquad s_D\approx0.548, \qquad t\approx5.72, \]
with 4 degrees of freedom and a two-sided p-value of approximately\[ \boxed{0.0046}. \]
If only the signs are used, all five differences are positive. The exact sign test gives:\[ \boxed{ p_{\text{one-sided}}=0.03125 } \]
and\[ \boxed{ p_{\text{two-sided}}=0.0625 }. \]
The normal approximation giving a two-sided p-value near 2.5% is unreliable for only five pairs. The exact binomial calculation should be used. The paired t-test is more powerful here because it uses the magnitudes of the age differences, while the sign test discards that information.