Multi-label Classification

1. Definition

• Multi-label classification = each instance (sample) can be assigned multiple labels simultaneously.
• Unlike multi-class classification (where exactly one label is chosen among many), here labels are not mutually exclusive.

$f: X \;\; \rightarrow \;\; \{0,1\}^K$

• For $K$ classes, each class has a binary decision (0 or 1) whether the instance belongs to it.

---

2. Examples

• Movies: A film can belong to multiple genres → Action + Comedy + Romance.
• News articles: An article can be tagged as Politics + Economy + International.
• Medical diagnosis: A patient may have multiple conditions (e.g., Diabetes + Hypertension).
• Image recognition: A picture may contain dog + car + tree.

---

3. How It Differs from Multi-class

Feature	Multi-class	Multi-label
Labels per sample	Exactly 1	One or more
Class exclusivity	Mutually exclusive	Not mutually exclusive
Typical output	Softmax (probabilities sum to 1)	Sigmoid (independent probability per class)
Example	Animal = {Cat, Dog, Horse} (one choice)	Tags = {Cat, Dog, Horse} (any combination)

---

4. Modeling

• Output layer:
  • Multi-class → Softmax activation (chooses one)
  • Multi-label → Sigmoid activation (thresholded independently for each label)
• Loss function:
  • Multi-class → Categorical cross-entropy
  • Multi-label → Binary cross-entropy (per class, summed/averaged)

---

5. Evaluation Metrics

Since predictions are multiple binary decisions per sample, standard metrics differ from multi-class:

• Per-label Precision, Recall, F1 (then averaged: micro, macro, weighted)
• Hamming loss (fraction of misclassified labels)
• Subset accuracy (strict: all labels correct = 1, else 0)
• Jaccard similarity (intersection over union of predicted vs true labels)

---

6. Example

Suppose true labels for an image: {Cat, Dog}
Model prediction: {Cat, Horse}

• Precision = 1 / (1+1) = 0.5 (only Cat correct, Horse is FP)
• Recall = 1 / (1+1) = 0.5 (missed Dog, so 1 FN)
• Jaccard = |{Cat} ∩ {Cat, Dog}| / |{Cat, Dog, Horse}| = 1/3 ≈ 0.33

---

Summary

• Multi-label classification allows assigning multiple labels to each sample.
• Labels are independent, not mutually exclusive.
• Requires sigmoid outputs and metrics like Hamming loss, Jaccard, and micro/macro F1.