Average Precision (AP)

Average precision summarizes a precision–recall curve by walking down the ranked predictions and recording the precision each time a real positive appears.

\(\text{AP} = \dfrac{1}{N_{\text{pos}}}\sum_{k=1}^{N} P(k)\cdot\mathbf{1}\{y_k = 1\}\)

where \(P(k)\) is the precision over the top \(k\) predictions. This form assumes the scores are distinct. Where several cases share a score there is no single ranked order, and listing the positives first gives a different answer from listing the negatives first — for one positive and one negative at the same score the row-by-row average is 1.0 while the threshold-based definition gives 0.5. The definition that handles ties is the threshold form scikit-learn documents, which treats all cases at one score as a single step and weights each precision by the gain in recall:

\(\text{AP} = \sum_n (R_n - R_{n-1})\,P_n\)

Because every positive contributes the precision at the moment it is found, positives ranked early contribute a high value and positives buried below many false alarms contribute a low one. A model can have high AP while an operating threshold chosen to keep only the top predictions has low recall. AP itself summarizes the complete ranked list, which reaches full recall when every case is included.

A worked case, and two other answers

import numpy as np
from sklearn.metrics import average_precision_score, precision_recall_curve, auc

labels = np.array([1, 0, 1, 1, 0])
scores = np.array([0.90, 0.80, 0.70, 0.60, 0.50])

hits = np.cumsum(labels)
by_hand = np.mean([hits[k] / (k + 1) for k in range(labels.size) if labels[k] == 1])
print(f"by hand  {by_hand:.4f}")
print(f"sklearn  {average_precision_score(labels, scores):.4f}")

precision, recall, _ = precision_recall_curve(labels, scores)
print(f"trapezoid under the same curve  {auc(recall, precision):.4f}")

rng = np.random.default_rng(23)
y = np.concatenate([np.ones(500), np.zeros(9500)])
s = np.concatenate([rng.normal(1.5, 1, 500), rng.normal(0, 1, 9500)])
p, r, _ = precision_recall_curve(y, s)
print(f"larger sample: AP {average_precision_score(y, s):.4f}  trapezoid {auc(r, p):.4f}")
# by hand  0.8056
# sklearn  0.8056
# trapezoid under the same curve  0.7639
# larger sample: AP 0.3466  trapezoid 0.3454

Five ranked predictions with labels 1, 0, 1, 1, 0 and five distinct scores. Positives appear at ranks 1, 3 and 4, with precisions of 1/1, 2/3 and 3/4; their mean is 0.8056, which is what average_precision_score returns. The two agree here because no two scores are equal; with ties only the threshold form is well defined.

The third line takes the trapezoidal area under the same precision–recall points and gets 0.7639. On the larger sample the two come out at 0.3466 and 0.3454. The numbers differ, the gap is not always in the same direction, and neither is a rounding error of the other — they are different definitions applied to the same curve.

What AP is not

AP is routinely described as a particular interpolation of the PR curve, usually one that builds a precision “envelope”. That is backwards. scikit-learn’s documentation is explicit: this implementation “is not interpolated and is different from computing the area under the precision-recall curve with the trapezoidal rule, which uses linear interpolation and can be too optimistic”. AP is the one that interpolates nothing.

The envelope version is the one the object-detection benchmarks use, and even there the details differ: PASCAL VOC originally sampled eleven recall levels, while COCO uses a 101-point interpolated AP averaged over ten IoU thresholds; both define their own metric and neither matches the other. “AP” in a detection paper and “AP” from average_precision_score are not the same quantity, which matters when numbers are compared across sources.

The objection to the trapezoid is not aesthetic. Davis and Goadrich show that “in PR space it is incorrect to linearly interpolate between points”: the straight line joining two achievable precision–recall points passes through combinations the classifier cannot produce, so the area under it credits performance that does not exist.

Reading the value

AP shares its baseline with PR-AUC: a random ranking scores approximately the prevalence of positives. So 0.20 is strong where positives are 2 percent of the data and below the random-ranking baseline where they are a quarter of it, and the number has to be reported with the prevalence to mean anything.

It is used where the top of a ranking is what gets consumed:

• Information retrieval. AP is the per-query summary that mean average precision averages across queries.
• Object detection. Reported per class and per IoU threshold, with the interpolated definition above.
• Rare-positive classification. Where ROC-AUC looks comfortable because the false positive rate divides by a very large number of negatives.

Like every ranking summary, AP is unchanged by any strictly increasing rescaling of the scores, so it cannot see calibration. And with few positives it is noisy — each one contributes \(1/N_{\text{pos}}\) of the total, so a single rank change near the top visibly moves the number. Report it with the prevalence, the number of positives, and which of the three definitions above you used.

References

• scikit-learn API reference, average_precision_score
• Jesse Davis and Mark Goadrich, “The Relationship Between Precision-Recall and ROC Curves”, ICML 2006
• Mark Everingham, Luc Van Gool, Christopher K. I. Williams, John Winn and Andrew Zisserman, “The Pascal Visual Object Classes (VOC) Challenge”, International Journal of Computer Vision, 88, 2010, 303–338
• COCO, Detection Evaluation