1. Binary AUROC Recap

  • AUROC = probability that the model ranks a randomly chosen positive sample higher than a randomly chosen negative sample.
  • Range: 0.5 (random) → 1.0 (perfect discrimination).

2. Multiclass AUROC Problem

When there are K classes, AUROC isn’t directly defined because ROC is inherently binary.
So, we extend it using one-vs-rest (OvR) or one-vs-one (OvO) strategies.

3. Macro AUROC Definition

  • Macro AUROC = average of AUROC values computed per class (treating each class as “positive” vs “all others”).
  • Formula:

$AUROC_{macro} = \frac{1}{K} \sum_{i=1}^{K} AUROC(\text{class}_i \; vs \; rest)$

  • Each class contributes equally, no matter how frequent or rare it is.

4. Macro vs Micro AUROC

  • Macro AUROC:
    • Equal weight for each class.
    • Sensitive to performance on minority classes.
  • Micro AUROC:
    • Flatten predictions across all classes, then compute AUROC globally.
    • Heavily influenced by majority classes.

5. Example

Suppose a 3-class classifier (A, B, C):

  • AUROC(A vs rest) = 0.85
  • AUROC(B vs rest) = 0.72
  • AUROC(C vs rest) = 0.65

Then:

$AUROC_{macro} = \frac{0.85 + 0.72 + 0.65}{3} = 0.74$

If class C is rare but performs poorly, Macro AUROC reflects that because each class counts equally.

6. Interpretation

  • Macro AUROC is best when you care about fairness across classes (balanced evaluation).
  • In highly imbalanced datasets, Macro AUROC highlights poor discrimination on small classes that Micro AUROC might hide.
  • Scale is the same as binary AUROC (0.5 random → 1.0 perfect).

Summary

  • Macro AUROC = average of per-class AUROC values (one-vs-rest).
  • Each class is weighted equally.
  • Useful when all classes matter equally, regardless of size.
  • Complements Micro AUROC, which focuses on overall sample-level performance.