Mann–Whitney U Test

The Mann–Whitney U test, also called the Wilcoxon rank-sum test, is a non-parametric test comparing two independent groups. It asks whether they come from the same distribution, and unlike the t-test it assumes nothing about normality — it works on the ranks of the values rather than the values.

Its assumptions are short: the two samples are independent, and the outcome is ordinal or continuous. A third is usually added as a footnote and matters more than it looks — if the two distributions have the same shape, the test can be read as a comparison of medians.

The statistic is built by pooling both groups, ranking everything, and summing the ranks within each group:

\[ U_1 = R_1 - \frac{n_1(n_1+1)}{2}, \qquad U_2 = R_2 - \frac{n_2(n_2+1)}{2}, \qquad U_1 + U_2 = n_1 n_2 \]

Some texts report \(U = \min(U_1, U_2)\) for the table lookup, and for large samples \(U\) is approximated by a normal distribution. The null hypothesis is that the two groups come from the same distribution; the alternative is that one tends to produce larger values.

Working the example, and the identity everyone quotes

import numpy as np
from scipy.stats import mannwhitneyu, rankdata, ttest_ind, norm, cauchy
from sklearn.metrics import roc_auc_score

A = np.array([88, 92, 100, 75, 85])
B = np.array([60, 70, 65, 80, 72])
n1, n2 = len(A), len(B)
ranks = rankdata(np.concatenate([A, B]))
R1, R2 = ranks[:n1].sum(), ranks[n1:].sum()
"""The exam-score example, worked out rather than described."""

U1 = R1 - n1 * (n1 + 1) / 2
U2 = R2 - n2 * (n2 + 1) / 2
print(f"  rank sums: group A {R1:.0f}, group B {R2:.0f}")
print(f"  U for A = R_A - n_A(n_A+1)/2 = {R1:.0f} - {n1 * (n1 + 1) / 2:.0f} = {U1:.0f}")
print(f"  U for B = R_B - n_B(n_B+1)/2 = {R2:.0f} - {n2 * (n2 + 1) / 2:.0f} = {U2:.0f}")
print(f"  they sum to n_A * n_B = {n1 * n2}: {U1 + U2:.0f}")
res = mannwhitneyu(A, B, alternative="two-sided")
print(f"  scipy agrees: U = {res.statistic:.0f}, p = {res.pvalue:.6f}")

labels = np.r_[np.ones(n1), np.zeros(n2)]
scores = np.concatenate([A, B])
print(f"    U_A / (n_A n_B)       {U1 / (n1 * n2):.4f}")
print(f"    roc_auc_score          {roc_auc_score(labels, scores):.4f}")
print(f"    min(U1, U2)/(n_A n_B)  {min(U1, U2) / (n1 * n2):.4f}   <- not the AUC")

rng = np.random.default_rng(0)
m = 400_000
x = np.where(rng.random(m) < 0.5, norm.ppf(rng.random(m)) * 0.3, norm.ppf(rng.random(m)) * 0.3 + 4)
y = norm.ppf(rng.random(m)) * 2.4 + 2.0
x = x - np.median(x) + 2.0
y = y - np.median(y) + 2.0
print(f"{'':>10}{'median':>10}{'mean':>10}{'P(X > Y)':>12}")
sub_x, sub_y = x[:4000], y[:4000]
print(f"{'X':>10}{np.median(x):>10.4f}{x.mean():>10.4f}", end="")
print(f"{np.mean(x[:200_000] > y[:200_000]):>12.4f}")
print(f"{'Y':>10}{np.median(y):>10.4f}{y.mean():>10.4f}")
u = mannwhitneyu(sub_x, sub_y, alternative="two-sided")
print(f"  Mann-Whitney on 4,000 of each: U = {u.statistic:.0f}, p = {u.pvalue:.2e}")

print(f"{'data':>22}{'t-test power':>15}{'Mann-Whitney power':>21}")
runs, n = 40_000, 30
for name, draw in (("normal, shift 0.8",
                    lambda s: norm.ppf(rng.random(s))),
                   ("Cauchy, shift 0.8",
                    lambda s: cauchy.ppf(rng.random(s)))):
    a = draw((runs, n))
    b = draw((runs, n)) + 0.8
    pt = ttest_ind(a, b, axis=1).pvalue
    pu = mannwhitneyu(a, b, axis=1, alternative="two-sided").pvalue
    print(f"{name:>22}{np.mean(pt < 0.05):>15.4f}{np.mean(pu < 0.05):>21.4f}")
#   rank sums: group A 39, group B 16
#   U for A = R_A - n_A(n_A+1)/2 = 39 - 15 = 24
#   U for B = R_B - n_B(n_B+1)/2 = 16 - 15 = 1
#   they sum to n_A * n_B = 25: 25
#   scipy agrees: U = 24, p = 0.015873
#     U_A / (n_A n_B)       0.9600
#     roc_auc_score          0.9600
#     min(U1, U2)/(n_A n_B)  0.0400   <- not the AUC
#               median      mean    P(X > Y)
#          X    2.0000    3.1089      0.6281
#          Y    2.0000    1.9957
#   Mann-Whitney on 4,000 of each: U = 9998910, p = 1.92e-83
#                   data   t-test power   Mann-Whitney power
#      normal, shift 0.8         0.8611               0.8396
#      Cauchy, shift 0.8         0.0548               0.3604

Three things the worked numbers settle

First, which \(U\). The identity between this statistic and ROC-AUC is exact, and it is exact for the \(U\) of the group being treated as positive: \(\text{AUC} = U_1 / (n_1 n_2)\). Feeding \(\min(U_1, U_2)\) into that formula gives 0.0400 where the AUC is 0.9600, because the minimum always sits below a half by construction. The two-sided p-value does not care which you report; the probability statement does.

With midranks for ties, \(U_1/(n_1 n_2)\) estimates \(P(X>Y)+\tfrac12P(X=Y)\). The standard Mann–Whitney null calibration assumes equal distributions. It need not maintain nominal level under every unequal-shape distribution pair whose probabilistic index happens to be 0.5. Testing that weaker null requires an appropriate method, such as a Brunner–Munzel procedure.

Second, it is not a median test. The second block builds two samples with the same median to four decimal places and the test rejects at \(p \approx 10^{-83}\), because \(P(X > Y) = 0.6281\) rather than 0.5. Nothing is broken — the test answered the question it asks. The equal-shape condition in the assumptions is what licenses the median reading, and it is a real condition rather than a formality. Where shapes differ and a median comparison is what you want, Mood’s median test or a bootstrap of the difference in medians answers that question directly.

Third, the trade against the t-test is not symmetric. On normal data the rank test gives up a little power — 0.8396 against 0.8611 in the third block, a few points for dropping the distributional assumption. On heavy-tailed data the t-test has essentially no power at all, 0.0548 against a nominal 0.05, while the rank test still detects the shift 0.3604 of the time. A single extreme value moves a mean and a standard deviation; it cannot move a rank past the end of the list.

When to reach for it

Use it when the outcome is ordinal, when the data are visibly skewed or heavy-tailed, or when only the relative ordering is meaningful — satisfaction scores, fully observed event times with a few enormous values; censored event times require an analysis that accounts for censoring, any metric where an outlier is real rather than a mistake.

Three cautions go with it. Ties reduce the effective information and need the tie correction that standard implementations apply by default. The output is a p-value about ordering, so report \(U/(n_1 n_2)\) alongside it as the effect size — it is interpretable on its own, being the probability of exceeding a random member of the other group plus half the probability of a tie. And a significant result says the distributions differ in the ordering sense; which feature differs is a separate question that the test does not answer.

References

• SciPy documentation, scipy.stats.mannwhitneyu and scipy.stats.rankdata. docs.scipy.org
• scikit-learn documentation, sklearn.metrics.roc_auc_score. scikit-learn.org