Macro Averaging

Macro averaging computes a metric separately for each class and takes the plain arithmetic mean. Every class counts once, whether it has ten thousand samples or fifty.

With per-class precisions of 0.90, 0.60 and 0.30, the macro precision is \((0.90 + 0.60 + 0.30) / 3 = 0.60\). The same procedure gives macro recall and macro AUROC, and it is the standard way to check that a model has not quietly abandoned a small class — which micro averaging will not show you.

Two details in the calculation are worth knowing before quoting the number.

Two macro F1s, and a keyword that moves the answer

import numpy as np
from sklearn.metrics import f1_score, precision_score, recall_score

y = np.array([0] * 700 + [1] * 250 + [2] * 50)
pred = y.copy()
rng = np.random.default_rng(0)
pred[rng.choice(np.flatnonzero(y == 0), 60, replace=False)] = 1
pred[rng.choice(np.flatnonzero(y == 1), 90, replace=False)] = 0
pred[np.flatnonzero(y == 2)] = 0
"""Class 2 is never predicted at all - the case macro averaging exists to catch."""

per_class_f1 = f1_score(y, pred, average=None, zero_division=0)
macro_p = precision_score(y, pred, average="macro", zero_division=0)
macro_r = recall_score(y, pred, average="macro", zero_division=0)
print("  per-class F1 " + "  ".join(f"class {k} {v:.4f}" for k, v in enumerate(per_class_f1)))
print(f"  macro precision {macro_p:.4f}   macro recall {macro_r:.4f}")
print(f"  mean of the per-class F1s (what sklearn calls macro F1) {per_class_f1.mean():.4f}")
print(f"  F1 computed from macro precision and macro recall        "
      f"{2 * macro_p * macro_r / (macro_p + macro_r):.4f}")

print("\n  the same data with zero_division set differently")
for setting in (0, 1):
    print(f"    zero_division={setting}   macro precision "
          f"{precision_score(y, pred, average='macro', zero_division=setting):.4f}"
          f"   macro F1 {f1_score(y, pred, average='macro', zero_division=setting):.4f}")
#   per-class F1 class 0 0.8649  class 1 0.6809  class 2 0.0000
#   macro precision 0.5159   macro recall 0.5181
#   mean of the per-class F1s (what sklearn calls macro F1) 0.5152
#   F1 computed from macro precision and macro recall        0.5170
#
#   the same data with zero_division set differently
#     zero_division=0   macro precision 0.5159   macro F1 0.5152
#     zero_division=1   macro precision 0.8493   macro F1 0.5152

“Macro F1” is ambiguous. You can average the per-class F1 scores — 0.5152 here, and what scikit-learn’s average='macro' returns — or you can compute an F1 from the macro precision and macro recall, which gives 0.5170. Both appear in the literature under the same name. They are not equal in general, because the harmonic mean of averages is not the average of harmonic means, and on data where per-class precision and recall diverge sharply the gap is much larger than it is here.

Nothing forces one choice. What matters is saying which one a number came from, especially when comparing against a published result.

A class the model never predicts has no precision. Class 2 is never predicted here, so its precision is zero true positives out of zero predictions — undefined. zero_division=0 scores it 0 and gives macro precision 0.5159; zero_division=1 scores it 1 and gives 0.8493. The predictions are identical in both rows.

Thirty-three points from a keyword argument is not a rounding concern. And the irony is sharp: macro averaging is the average you reach for to detect an abandoned class, and the most charitable setting reports that abandoned class as perfect. Set it to 0 unless you have a reason, and state the setting in the report.

Recall and F1 for class 2 are both well-defined zeros here because it has true instances but no predicted positives. Only its precision needs a zero-division convention. F1 becomes undefined when TP, FP and FN are all zero, so explicitly including an entirely absent class is a different case.

How hard it actually penalizes

Macro averaging is described as penalizing poor performance on minority classes, and it does — but the penalty is \(1/K\) of the total, not proportional to how much the class matters.

With three classes, one class scoring 0 costs a third of the macro score, which is loud. With a hundred classes, a class scoring 0 costs one point of a hundred, which will not be noticed in a dashboard. So on large label sets macro averaging is not the alarm it is on small ones, and the per-class table is what finds the problem.

The converse is also worth stating. Macro averaging asserts that every class matters equally, and that is a claim about the application, not a neutral default. If one class is a rare disease and another is a common cold, equal weight is a position someone should have taken deliberately.

Choosing among the three

• Macro — one vote per class. Use it when classes are equally important, and when you want a small class’s failure to show.
• Micro — one vote per sample. In single-label problems it equals accuracy exactly, so quoting both says one thing twice.
• Weighted — one vote per sample, applied to per-class metrics. Between the two, with a quirk of its own: the weighted F1 need not lie between the weighted precision and recall.

Report macro and micro with the per-class results and supports. Their difference can reveal a consequence of aggregation, but it does not measure class disparity on its own. With two equally supported classes whose recalls are 1.0 and 0.5, macro and micro recall are both 0.75 despite the unequal performance. For pooled AUC, cross-class score scales add another source of differences.

References

• scikit-learn API reference, f1_score
• scikit-learn User Guide, Metrics and scoring