Average Precision (AP)
1) What AP measures
- AP is a ranking-based metric that summarizes the Precision–Recall (PR) curve.
- It tells you how well your model ranks true positives ahead of false positives, with more weight on higher recall levels.
In plain terms:
- AP = “average of precision values measured whenever you discover a new true positive as you move down the ranked list.”
- Higher AP → more true positives are found early in the ranking with high precision.
2) How AP is computed
Suppose we sort predictions by score (highest → lowest).
- Sweep threshold down the ranked list.
- Each time you encounter a new true positive, compute precision at that point.
- Average those precisions over all true positives.
Formula:
$AP = \frac{1}{N_{pos}} \sum_{k=1}^{N} P(k) \cdot \mathbf{1}\{y_k = 1\}$
- $N_{pos}$: total number of positives
- $P(k)$: precision at rank $k$
- $y_k$: true label at position $k$
This is equivalent to integrating the precision–recall curve using a specific interpolation method.
3) Difference from PR-AUC
- PR-AUC: Area under the PR curve, usually trapezoidal interpolation.
- AP: Uses a step-wise “envelope” interpolation (only precision at positive instances counts).
They are close but not identical.
Scikit-learn’s average_precision_score reports AP, not trapezoidal PR-AUC.
4) Example
Say we have 5 ranked predictions:
| Rank | True Label | Precision at this rank |
|---|---|---|
| 1 | 1 | 1/1 = 1.0 |
| 2 | 0 | (skip, not positive) |
| 3 | 1 | 2/3 ≈ 0.67 |
| 4 | 1 | 3/4 = 0.75 |
| 5 | 0 | (skip) |
Total positives = 3.
$AP = (1.0 + 0.67 + 0.75)/3 = 0.81$
5) Interpretation
- AP = 1.0 → perfect ranking (all positives ranked above negatives).
- AP close to baseline → model is poor.
- Baseline AP = positive class proportion.
- Example: if positives = 5%, then random guessing gives AP ≈ 0.05.
6) Why AP is popular
- Information retrieval: AP = average precision at all recall levels → matches ranking tasks.
- Object detection (COCO, PASCAL VOC): AP at multiple IoU thresholds is the gold standard metric.
- Class imbalance: More sensitive than AUROC when positives are rare.
Summary
- AP = average precision at the ranks of all positives.
- Closely related to PR-AUC, but uses a different interpolation (precision envelope).
- Strongly affected by class imbalance and ranking quality.
- A good metric when you care about finding positives early in the ranked list.