Uncertainty Estimation and Model Calibration

Softmax outputs are probability estimates, but an output of 0.99 does not by itself establish a 99% chance of correctness. Calibration checks whether such estimates agree with outcome frequencies on a specified population. Uncertainty estimation asks additional questions about ambiguity and model disagreement; prediction sets address yet another question, how often a set contains the true label.

What calibration means

For top-label calibration, confidence \(c(x)=\max_k p_k(x)\) should match \(P(\hat Y=Y\mid c(X)=s)=s\). Among predictions with confidence near 0.8, approximately 80% should be correct in a sufficiently large sample. Calibration and accuracy measure different properties, rather than being statistically independent: exact top-label calibration implies that mean confidence equals accuracy. Classwise calibration additionally checks each class probability against the frequency of that class, and can fail even when top-label calibration holds.

For a finite sample, expected calibration error (ECE) groups predictions into bins \(B_b\). Each bin contributes its fraction of the sample times the absolute difference between mean correctness and mean confidence. The common name “expected” does not make this binned statistic an exact population quantity:

\[\text{ECE}=\sum_{b=1}^{B}\frac{|B_b|}{n}\left|\text{acc}(B_b)-\text{conf}(B_b)\right|\]

import numpy as np

def ece(confidences, correct, n_bins=10):
    c = np.asarray(confidences, dtype=float)
    y = np.asarray(correct)
    if c.ndim != 1 or y.shape != c.shape or len(c) == 0:
        raise ValueError("Expected nonempty, equal-length vectors")
    if not np.isfinite(c).all() or np.any((c < 0) | (c > 1)) or not np.isin(y, [0, 1]).all():
        raise ValueError("Invalid confidence or correctness")
    if not isinstance(n_bins, (int, np.integer)) or n_bins < 1:
        raise ValueError("n_bins must be a positive integer")
    idx = np.minimum((c*n_bins).astype(int), n_bins-1)
    total = 0.0
    for b in range(n_bins):
        mask = idx == b
        if mask.any():
            total += mask.mean()*abs(y[mask].mean()-c[mask].mean())
    return total

rng = np.random.default_rng(0)
p_true = rng.uniform(0.5, 1.0, 5000)               # calibrated confidence process
correct = (rng.uniform(size=5000) < p_true).astype(float)
print("calibrated  ECE:", round(ece(p_true, correct), 4))

overconf = np.clip(p_true ** 0.4, 0, 1)            # same accuracy, inflated
print("overconfident ECE:", round(ece(overconf, correct), 4))
# calibrated  ECE: 0.0132
# overconfident ECE: 0.1457

This simulation holds correctness labels fixed and changes only reported confidence; it does not train two classifiers. A calibrated generating process still has nonzero empirical ECE because outcomes fluctuate. A reliability diagram plots each nonempty bin’s mean confidence against its empirical accuracy, preferably with bin counts and uncertainty. Points below the diagonal indicate overconfidence within those bins.

Absolute gaps from different bins do not cancel. Opposing errors can cancel within a bin, however, and group differences can disappear after aggregation: two equally sized groups reporting 0.8 confidence with accuracies 1.0 and 0.6 give aggregate accuracy 0.8. ECE also ignores non-top probabilities and depends on binning and sample size. Report the binning protocol, reliability information, and proper scoring losses such as negative log-likelihood or Brier loss, which assess additional aspects of the predictive distribution.

Deciding what to improve next is the subject of A Systematic Strategy for Improving Machine Learning Models.

How confidence can become miscalibrated

For separable training examples, an unregularized cross-entropy fit can keep reducing empirical loss by increasing correct-class logit margins after labels are classified correctly. That behavior can contribute to overconfidence on new data. It is not a property of the population optimum of cross-entropy, which favors the true conditional probabilities when representable. Architecture, regularization, training procedure, and distribution shift affect the observed result.

Label smoothing, mixup, and weight decay can alter calibration, sometimes improving it and sometimes producing underconfidence or other trade-offs. Their effect must be measured on representative held-out data. They do not all impose a hard bound on logits or guarantee calibrated probabilities.

Temperature scaling

Temperature scaling adjusts a fixed classifier using one positive scalar:

\[\hat p=\text{softmax}(z/T)\]

For finite logits and a positive scalar \(T\), division preserves class ordering in exact arithmetic, so top-1 predictions are unchanged. \(T>1\) softens the distribution and \(0<T<1\) sharpens it. Thresholded decisions, abstention, and cross-example confidence rankings can still change. Fit \(T\) by minimizing negative log-likelihood on calibration data, then evaluate on separate test data; there is no universal temperature range.

import torch, torch.nn as nn, torch.nn.functional as F
from scipy.optimize import minimize_scalar

def fit_temperature(logits, labels):
    z = logits.detach().to(device="cpu", dtype=torch.float64)
    y = labels.detach().to(device="cpu")
    if z.ndim != 2 or z.shape[0] == 0 or z.shape[1] < 2 or y.shape != (len(z),):
        raise ValueError("Expected logits (n,C) and labels (n,)")
    if y.dtype != torch.long or not torch.isfinite(z).all() or not ((y >= 0) & (y < z.shape[1])).all():
        raise ValueError("Invalid logits or class indices")
    def objective(log_t):
        return F.cross_entropy(z / np.exp(log_t), y).item()
    result = minimize_scalar(objective, bounds=(-4, 4), method="bounded")
    if not result.success or abs(result.x) > 3.99:
        raise RuntimeError("Temperature search failed or reached its chosen range")
    return float(np.exp(result.x))

def synthetic_logits(n, generator):
    latent = torch.randn(n, 5, generator=generator, dtype=torch.float64)
    labels = torch.multinomial(latent.softmax(-1), 1, generator=generator).squeeze(1)
    return 3*latent, labels

generator = torch.Generator().manual_seed(0)
cal_logits, cal_labels = synthetic_logits(2000, generator)
test_logits, test_labels = synthetic_logits(4000, generator)
T = fit_temperature(cal_logits, cal_labels)
print(f"T={T:.3f}")
for tag, z in (("before", test_logits), ("after", test_logits/T)):
    confidence, prediction = z.softmax(-1).max(-1)
    correct = prediction == test_labels
    print(f"{tag} NLL={F.cross_entropy(z,test_labels).item():.4f} ECE={ece(confidence.numpy(),correct.numpy()):.4f} accuracy={correct.double().mean():.4f}")
print("argmax unchanged", torch.equal(test_logits.argmax(-1),(test_logits/T).argmax(-1)))
# T=3.077
# before NLL=1.7755 ECE=0.2764 accuracy=0.4693
# after NLL=1.2921 ECE=0.0178 accuracy=0.4693
# argmax unchanged True

The helper detaches logits and performs a one-dimensional CPU search over log T in [-4,4]; T is positive, while log T can have either sign. A boundary solution is reported instead of silently presented as a well-determined optimum. The generator deliberately inflates logits by three, so a temperature near three has a known interpretation here. In this run T is 3.077; test NLL falls from 1.7755 to 1.2921 and ECE from 0.2764 to 0.0178, with accuracy unchanged at 0.4693. This synthetic example is not a trained-network benchmark. Fitting on training data does not generally force T near one; the concern is that training predictions do not provide independent evidence of calibration on new data.

Two kinds of uncertainty

Aleatoric uncertainty refers to outcome variability conditional on the available information. More observations of the same kind can estimate that variability better without removing it; additional informative features or better measurement can change it. Epistemic uncertainty concerns uncertainty about the model or its parameters given data and assumptions. Relevant new data can reduce it, but simply increasing dataset size does not ensure that a misspecified learner resolves its errors.

These terms suggest different investigations: inspect measurement quality and ambiguous outcomes, or seek evidence in poorly supported regions. They do not by themselves determine whether human review will help. High epistemic uncertainty need not mean an input is out of distribution, and unfamiliar inputs can receive low estimated uncertainty. The approximations below measure model behavior under particular constructions, not an assumption-free separation of the two sources.

Deep ensembles

An ensemble averages probability vectors from separately trained models, often with different initialization and data-order seeds. Shared data and inductive biases can make their errors correlated. Disagreement is useful evidence about variation among those models, but members can agree and be wrong on unfamiliar inputs.

def ensemble_stats(prob_list):
    P = np.asarray(prob_list, dtype=float)
    if P.ndim != 3 or min(P.shape) < 1 or not np.isfinite(P).all() or np.any(P < 0):
        raise ValueError("Expected finite probabilities (M,n,C)")
    if not np.allclose(P.sum(-1), 1):
        raise ValueError("Probability rows must sum to one")
    entropy = lambda q: -(q*np.log(np.clip(q, 1e-300, 1))).sum(-1)
    mean = P.mean(0)
    total = entropy(mean)
    expected_member_entropy = entropy(P).mean(0)
    return mean, total, expected_member_entropy, total-expected_member_entropy

agree = [np.array([[.8,.2]]), np.array([[.79,.21]]), np.array([[.81,.19]])]
split = [np.array([[.99,.01]]), np.array([[.02,.98]]), np.array([[.97,.03]])]
for name, probabilities in [("agreeing",agree),("disagreeing",split)]:
    _, total, within, disagreement = ensemble_stats(probabilities)
    print(f"{name} total={total[0]:.4f} member-entropy={within[0]:.4f} disagreement={disagreement[0]:.4f}")
# agreeing total=0.5004 member-entropy=0.5002 disagreement=0.0002
# disagreeing total=0.6410 member-entropy=0.0963 disagreement=0.5448

Entropy \(H(p)=-\sum_c p_c\log p_c\) is measured in nats here. The gap \(H(\bar p)-M^{-1}\sum_m H(p_m)\) is nonnegative by concavity and equals the mutual information between a uniformly chosen member index and its sampled label. Interpreting it as epistemic uncertainty, and the member entropy as aleatoric uncertainty, requires a suitable model-distribution interpretation. The hand-built probability arrays demonstrate this calculation only. Training and inference work generally grows with the number of members, while wall time and memory depend on parallelism and deployment.

MC dropout and cheaper approximations

Repeated forward passes with dropout active produce a distribution of predictions from one fitted network. Under a particular dropout training and variational interpretation, this can approximate Bayesian inference; arbitrary inference-time dropout is not automatically a posterior. It uses one training run and k forward passes. Compared with k ensemble members, inference work can be similar, although parameter storage and training requirements differ.

@torch.no_grad()
def mc_dropout_predict(model, x, k=30):
    if not isinstance(k, int) or k < 2:
        raise ValueError("k must be at least 2")
    flags = [(module, module.training) for module in model.modules()]
    dropout_types = (nn.Dropout, nn.Dropout1d, nn.Dropout2d, nn.Dropout3d, nn.AlphaDropout, nn.FeatureAlphaDropout)
    try:
        model.eval()
        for module in model.modules():
            if isinstance(module, dropout_types):
                module.train()
        probabilities = torch.stack([model(x).softmax(-1) for _ in range(k)])
        return probabilities.mean(0), probabilities.var(0, unbiased=False).sum(-1)
    finally:
        for module, flag in flags:
            module.training = flag

torch.manual_seed(0)
toy = nn.Sequential(nn.Linear(4,8), nn.BatchNorm1d(8), nn.ReLU(), nn.Dropout(.5), nn.Linear(8,3))
x = torch.randn(6,4)
flags_before = [module.training for module in toy.modules()]
bn_before = toy[1].running_mean.clone()
mean, variance = mc_dropout_predict(toy, x)
print("output shapes", tuple(mean.shape), tuple(variance.shape))
print("positive variation", bool((variance > 0).all()))
print("BatchNorm mean unchanged", torch.equal(bn_before, toy[1].running_mean))
print("training flags restored", flags_before == [module.training for module in toy.modules()])
# output shapes (6, 3) (6,)
# positive variation True
# BatchNorm mean unchanged True
# training flags restored True

The helper first uses evaluation behavior and then enables standard dropout modules only, restoring every saved training flag even on failure. This avoids updating BatchNorm running statistics. Functional dropout, dropout internal to other modules, and custom stochastic layers need model-specific handling. The untrained toy checks execution and state preservation, not uncertainty quality. Probability variance here is the variance across the k sampled predictions using divisor k, not an entropy decomposition.

Conformal prediction

Split conformal classification forms a label set using a fixed score function and a separate conformal calibration sample. Under exchangeability of calibration examples and a fresh test example, the construction below guarantees \(P(Y_{\rm new}\in C(X_{\rm new}))\ge1-\alpha\), marginally over calibration and test randomness. “Distribution-free” does not mean assumption-free. Keep the predictive model, temperature, and score definition fixed before using this calibration sample.

def conformal_threshold(cal_probs, cal_labels, alpha=0.1):
    P = np.asarray(cal_probs, dtype=float); y = np.asarray(cal_labels)
    if P.ndim != 2 or len(P) == 0 or P.shape[1] < 2 or y.shape != (len(P),):
        raise ValueError("Expected probabilities (n,C) and labels (n,)")
    if not 0 < alpha < 1 or not np.isfinite(P).all() or np.any(P < 0) or not np.allclose(P.sum(1),1):
        raise ValueError("Invalid probabilities or alpha")
    if not np.issubdtype(y.dtype,np.integer) or np.any((y<0)|(y>=P.shape[1])):
        raise ValueError("Invalid class indices")
    scores = 1-P[np.arange(len(y)),y]
    rank = int(np.ceil((len(scores)+1)*(1-alpha)))
    if rank > len(scores):
        return np.inf
    return np.sort(scores)[rank-1]

def prediction_set(probs, qhat):
    return [np.where(1 - p <= qhat)[0] for p in probs]

rng = np.random.default_rng(0)
cal = rng.dirichlet(np.ones(5) * 0.6, size=2000)
cal_y = np.array([rng.choice(5, p=p) for p in cal])
qhat = conformal_threshold(cal, cal_y, alpha=0.1)

test = rng.dirichlet(np.ones(5) * 0.6, size=2000)
test_y = np.array([rng.choice(5, p=p) for p in test])
sets = prediction_set(test, qhat)
cov = np.mean([y in s for y, s in zip(test_y, sets)])
print(f"qhat={qhat:.3f} coverage={cov:.3f} mean set size={np.mean([len(s) for s in sets]):.2f}")
# qhat=0.882 coverage=0.894 mean set size=2.64

Using nonconformity \(1-p_y\), choose the \(k\)-th smallest calibration score with \(k=\lceil(n+1)(1-\alpha)\rceil\). For n=9 and alpha=0.1, k=9: use the maximum. If k exceeds n, infinity gives the full label set and preserves the guarantee. Clamping an out-of-range quantile to the observed maximum would not suffice. This rank construction handles ties conservatively. The model need not be calibrated, but efficiency depends on the score and task; a worse model does not necessarily give larger sets on every input.

A sudden change in set sizes can motivate investigation, but ordinary sampling variation and model updates can also change them. Pure conditional label drift with unchanged inputs and a fixed model can leave sets unchanged while coverage deteriorates. Monitoring and outcome-based checks are discussed in Monitoring, Drift, and Retraining.

Choosing a method

Method	Fitting or calibration	Prediction work	Output and limitation
Temperature scaling	fit one scalar on held-out logits	logit rescaling	adjusted probabilities; no guarantee of perfect calibration
Deep ensemble	train M members	M model evaluations	average probabilities and disagreement; shared errors remain
MC dropout	a suitable dropout-trained model	k stochastic passes	variation under dropout masks; interpretation depends on assumptions
Split conformal	separate score-calibration sample	base prediction plus class-set construction	marginal coverage under exchangeability and a fixed score rule

Choose the output required by the decision: probability estimates, disagreement diagnostics, or a prediction set with marginal coverage. If combining temperature scaling and split conformal, use separate temperature-fitting and conformal-calibration data, followed by untouched evaluation data. Reusing conformal labels to tune temperature or select the scoring method can invalidate the standard split-conformal argument.

For selective prediction, report the fraction of cases accepted and the error among accepted cases across thresholds, together with the cost and quality of the fallback. In multiclass temperature scaling, confidence rankings across examples can change even though each argmax stays fixed. Low confidence can be monitored as a signal, but high confidence does not certify correctness or absence of distribution shift.

The snippets are intended to run after the main examples so their helpers are defined.

References

• Guo et al., On Calibration of Modern Neural Networks.
• Gal and Ghahramani, Dropout as a Bayesian Approximation.
• Angelopoulos and Bates, A Gentle Introduction to Conformal Prediction.
• Lakshminarayanan, Pritzel, and Blundell (2017). Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles.