F1-score

The F1-score is the harmonic mean of precision and recall:

\(F_1 = 2 \cdot \dfrac{\text{precision} \cdot \text{recall}}{\text{precision} + \text{recall}}\)

It exists because precision and recall are each trivially gamed on their own — flag one certain case and precision is 1.0; flag everything and recall is 1.0 — and a single number that neither trick can raise is useful.

Why the harmonic mean

An arithmetic mean lets one high value hide a low one. Precision 1.0 with recall 0.01 averages to 0.505, even though only one percent of positives were found. The harmonic mean is about 0.0198, and more generally it sits closer to the smaller of the two numbers the further apart they are.

What it leaves out

Write F1 in terms of the confusion matrix and one cell is missing:

\(F_1 = \dfrac{2\,TP}{2\,TP + FP + FN}\)

There is no \(TN\). This is usually presented as a virtue — it is why F1 survives class imbalance where accuracy does not — and it is, but it has a consequence that is less often stated. A metric that ignores one cell cannot be symmetric in the two classes, so F1 depends on which class you decided to call positive.

import numpy as np

def prf(tp, fp, fn):
    precision = tp / (tp + fp)
    recall = tp / (tp + fn)
    return precision, recall, 2 * precision * recall / (precision + recall)

TP, FP, FN, TN = 50, 20, 10, 920
"""One confusion matrix from 1,000 emails, with spam as the positive class."""

p, r, f1 = prf(TP, FP, FN)
print(f"spam positive      precision {p:.4f}  recall {r:.4f}  F1 {f1:.4f}")
p2, r2, f1_swapped = prf(TN, FN, FP)
print(f"not-spam positive  precision {p2:.4f}  recall {r2:.4f}  F1 {f1_swapped:.4f}")
print(f"accuracy is the same either way: {(TP + TN) / (TP + FP + FN + TN):.4f}")

print()
for precision, recall in ((0.9, 0.9), (0.95, 0.85), (1.0, 0.8), (1.0, 0.5), (1.0, 0.01)):
    harmonic = 2 * precision * recall / (precision + recall)
    print(f"precision {precision:.2f}  recall {recall:.2f}  "
          f"arithmetic {(precision + recall) / 2:.4f}  F1 {harmonic:.4f}")
# spam positive      precision 0.7143  recall 0.8333  F1 0.7692
# not-spam positive  precision 0.9892  recall 0.9787  F1 0.9840
# accuracy is the same either way: 0.9700
#
# precision 0.90  recall 0.90  arithmetic 0.9000  F1 0.9000
# precision 0.95  recall 0.85  arithmetic 0.9000  F1 0.8972
# precision 1.00  recall 0.80  arithmetic 0.9000  F1 0.8889
# precision 1.00  recall 0.50  arithmetic 0.7500  F1 0.6667
# precision 1.00  recall 0.01  arithmetic 0.5050  F1 0.0198

One confusion matrix, two answers: 0.7692 with spam as the positive class, 0.9840 with not-spam as the positive class. Accuracy is 0.9700 either way, because it uses all four cells and is symmetric by construction.

Neither F1 is wrong. They answer different questions, and the labeling convention picks which one gets reported. The practical rule is that F1 belongs to the class you actually care about — the rare one, the one with the expensive errors — and a reported F1 is incomplete without saying which class it was computed for.

The second half of that output shows the harmonic mean at work. Three precision–recall pairs with an identical arithmetic mean of 0.9000 give F1 scores of 0.9000, 0.8972 and 0.8889; the more lopsided the pair, the larger the gap. At precision 1.0 and recall 0.01 the arithmetic mean still reads 0.5050 while F1 has collapsed to 0.0198.

Weighting one side

\(F_\beta\) generalizes the same idea with a weight:

\(F_\beta = (1 + \beta^2) \cdot \dfrac{\text{precision} \cdot \text{recall}}{\beta^2 \cdot \text{precision} + \text{recall}}\)

\(\beta > 1\) favors recall, \(\beta < 1\) favours precision, and \(\beta = 1\) is F1. \(F_2\) and \(F_{0.5}\) are the conventional choices.

Choosing \(\beta\) sets the relative emphasis on precision and recall. It is not a one-to-one encoding of fixed false-positive and false-negative costs. If the costs are known in money or in time, computing the threshold directly from them is better founded — see recalibrate thresholds. \(F_\beta\) is for when the asymmetry is real but cannot be priced, and someone still has to pick a single number to optimize.

Multiclass averaging

Scheme	How it is computed	What it favors
Macro	F1 per class, then an unweighted mean	Rare classes, which count as much as common ones
Micro	Pool \(TP\), \(FP\), \(FN\) across classes, then one F1	Common classes. In single-label multiclass this equals accuracy
Weighted	F1 per class, averaged by support	Common classes, while still reporting per-class behavior

The micro row is worth knowing before choosing it: in a single-label problem every mistake is simultaneously a false positive for one class and a false negative for another, so the pooled counts collapse and micro F1 is accuracy under another name. If accuracy was rejected for this problem, micro F1 does not fix anything.

When not to use it

• When only one error type matters. Report that metric directly rather than blending it with one nobody is asking about.
• When the probabilities themselves are the product. F1 is computed after a threshold and says nothing about whether 0.9 means 0.9 — use Brier score or log loss.
• When comparing models across operating points. F1 is one point on a precision–recall curve, and two models can trade places depending on where the threshold sits. Compare the curves, or the ROC curve, and pick the point afterward.
• When the true negatives matter. Specificity and accuracy see them; F1 does not.

References

• Takaya Saito and Marc Rehmsmeier, “The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets”, PLOS ONE, 10(3), 2015
• scikit-learn API reference, f1_score