Multiclass AUROC

1. Binary AUROC Recap

• In binary classification, AUROC measures how well the model separates positive vs negative classes.
• Defined as the probability that a randomly chosen positive instance is scored higher than a randomly chosen negative instance.
• AUROC = 0.5 (random), AUROC = 1.0 (perfect).

---

2. Challenge in Multiclass

• In multiclass (more than 2 classes), we don’t have just one “positive vs negative” situation.
• Instead, for K classes, each class can be treated as “positive” vs “all other classes” (“One-vs-Rest”).
• So we need strategies to extend AUROC beyond binary.

---

3. Two Main Approaches

(a) One-vs-Rest (OvR) AUROC

• Compute AUROC for each class vs all others.
• Example: for 3 classes (A, B, C):
  • AUROC(A vs not A)
  • AUROC(B vs not B)
  • AUROC(C vs not C)
• Then average these AUROCs:
  • Macro-Averaged AUROC = unweighted mean across classes
  • Weighted-Averaged AUROC = weighted by class frequency

(b) One-vs-One (OvO) AUROC

• Compute AUROC for every pair of classes.
• Example: for 3 classes → (A vs B), (A vs C), (B vs C).
• Average across all pairs.
• This method gives a more balanced view when class distributions are highly imbalanced.

---

4. Micro vs Macro Averaging

• Micro-Averaged AUROC: Treat all samples equally, flatten into binary decisions, then compute AUROC.
  • Best when class imbalance exists.
• Macro-Averaged AUROC: Compute per-class AUROC, then average.
  • Best when we want each class treated equally.

---

5. Formulas

• OvR approach (macro average):

$AUROC_{macro} = \frac{1}{K} \sum_{i=1}^{K} AUROC(class_i \; vs \; rest)$

• OvO approach (macro average):

$AUROC_{ovo} = \frac{2}{K(K-1)} \sum_{i < j} AUROC(class_i \; vs \; class_j)$

---

6. Example

Suppose we have 3 classes (A, B, C).

• AUROC(A vs rest) = 0.82
• AUROC(B vs rest) = 0.75
• AUROC(C vs rest) = 0.70

Then:

• Macro AUROC = (0.82 + 0.75 + 0.70) / 3 = 0.7567
• Weighted AUROC depends on class sizes.

---

7. Interpretation

• Same scale as binary AUROC (0.5 random, 1.0 perfect).
• Higher values mean better discrimination across classes.
• Macro AUROC: fairness across all classes
• Micro AUROC: global performance across all samples
• OvO AUROC: pairwise discrimination ability

---

Summary

• Multiclass AUROC extends binary AUROC using One-vs-Rest or One-vs-One.
• Report usually includes macro, micro, and sometimes weighted averages.
• Each method answers slightly different performance questions (balanced vs imbalanced data, pairwise vs overall separation).