Probabilistic Scoring
Three forecasters saying 0.70, 0.80 and 0.99 are correct on exactly the same 70% of days.
-
-
Strictly Proper Scoring Rules
The property that makes a probability metric ungameable: honest reporting is uniquely optimal. Shown by calculation, including a rule that lacks it.
-
Log Loss
The negative log-likelihood, used as a metric. Strictly proper, unbounded, and sensitive enough to a single confident error to need watching.
-
Brier Score
Mean squared error for probabilities. It rewards calibration and discrimination at once, which is why it needs a baseline before it means anything.