Micro AUROC

Micro AUROC flattens a multi-class problem into one binary one. Each sample contributes \(K\) rows — one per class, each asking “does this sample belong to class \(k\)?” — and the pooled label and score columns are scored once with an ordinary AUROC.

The inputs are pairs, not counts: the pooled set is \(\{(y_{ik}, p_{ik})\}\) over every sample \(i\) and class \(k\), where \(y_{ik}\) is 1 when sample \(i\) is in class \(k\). Scoring it answers one question — across the whole pool, does a true (sample, class) pairing get a higher score than a false one?

It compares one class’s scores against another’s

That pooling is the whole of it, and it carries an assumption that macro AUROC does not make. A pooled comparison can put a class-0 score next to a class-2 score, so the answer depends on whether the two columns are on the same scale.

import numpy as np
from sklearn.metrics import roc_auc_score

rng = np.random.default_rng(3)
classes, n = 3, 3000
y = rng.choice(classes, size=n, p=[0.7, 0.2, 0.1])
scores = rng.normal(size=(n, classes))
for k in range(classes):
    scores[y == k, k] += 1.5
probs = np.exp(scores) / np.exp(scores).sum(axis=1, keepdims=True)
onehot = np.eye(classes)[y]

def micro(p):
    """Pool every one-vs-rest decision into one binary problem, then score it once."""
    return roc_auc_score(onehot.ravel(), p.ravel())

sharpened = probs.copy()
sharpened[:, 2] = sharpened[:, 2] ** 3
for label, p in (("as fitted", probs), ("class 2 cubed", sharpened)):
    per_class = [roc_auc_score(onehot[:, k], p[:, k]) for k in range(classes)]
    print(f"{label:14} per-class " + " ".join(f"{a:.6f}" for a in per_class)
          + f"  macro {np.mean(per_class):.6f}  micro {micro(p):.6f}")
# as fitted      per-class 0.911209 0.921486 0.929220  macro 0.920638  micro 0.918263
# class 2 cubed  per-class 0.911209 0.921486 0.929220  macro 0.920638  micro 0.934812

Cubing one column is a strictly increasing transformation of that class’s scores, so it cannot change any ranking within the class. The per-class values confirm it — 0.911209, 0.921486, 0.929220 both times, identical to six decimal places, and macro AUROC with them. Micro AUROC moves from 0.918263 to 0.934812.

Nothing about the model’s ability to recognize any class has changed. What changed is how class 2’s numbers sit against classes 0 and 1 in a shared ordering, and micro AUROC is measuring that. On scores that are not calibrated across classes, the metric is partly reading an accident of scale.

Softmax outputs are at least on a common footing — they sum to one per sample — which is why micro AUROC is usually computed on them rather than on raw per-class scores. That makes them comparable, not correct: see calibration.

Where the majority classes come in

Micro averaging is routinely described as weighting by sample frequency, and for a single-label problem that is not quite the mechanism. Every sample contributes exactly \(K\) rows, so samples are weighted equally. What is not equal is the composition of the positive rows: a class holding 70 percent of the data supplies 70 percent of them.

The consequence is the familiar one — a rare class that the model ranks badly contributes few positives and barely moves the number — but the reason is worth getting right, because it changes in the multilabel case. There a sample can carry several labels. With all K labels evaluated, each sample still supplies K entries, but samples supply different numbers of positives and negatives to the pooled ranking comparisons.

Reporting it

• Report macro and per-class results alongside. A gap can prompt an investigation of class supports and performance, but it can also reflect cross-class score scales, as the experiment above shows. It is not by itself proof that rare classes perform worse.
• Say which scores were pooled. Softmax probabilities, raw logits and independently fitted per-class scores give three different numbers.
• Do not read it as a per-class result. It is a statement about one pooled ranking, and no class’s performance can be recovered from it.
• scikit-learn supports roc_auc_score(y, probabilities, multi_class="ovr", average="micro") for multiclass input. The probability columns must follow the class order and each row must sum to one. Flattening the binarized labels and matching score matrix is another way to compute the pooled binary ranking; it is also useful for illustrating arbitrary per-class scores that do not meet the multiclass probability-input contract. OvO does not provide this micro average.

For the separate question of whether “AUC” here means the area under a ROC curve or a precision–recall curve — two numbers with different baselines — see micro AUC.

References

• scikit-learn API reference, roc_auc_score
• Tom Fawcett, “An introduction to ROC analysis”, Pattern Recognition Letters, 27(8), 2006, 861–874