Off-Distribution

An off-distribution input is one that sits outside the region the model was trained on. The term is usually introduced alongside out-of-distribution as a synonym, and alongside distribution shift as a broader version of the same idea. The second identification is the one to break: a batch can shift as a population without containing a single unusual row, and a handful of unusual rows can arrive without the batch as a whole looking shifted.

One row or one batch

Row-level novelty detection and batch-level drift monitoring answer different questions. OOD is also used for distributions, populations and tasks; here it means a single input outside the training region. Individually familiar rows can change their mixture, and a few unusual rows can be hard for a batch test to detect.

The row-level detector is a distance gate. A row counts as novel when its mean distance to its ten nearest training rows is larger than the 99.9th percentile of that same quantity measured among the training rows themselves, excluding each row from its own neighbors. novel_rate is the share of a batch past that gate. The training data is two Gaussian modes, and mixture(w, ...) puts a fraction w of the rows in the shifted mode. Three current batches are tested: a fresh draw from the training distribution, the same two modes with the shifted mode’s weight moved from 0.50 to 0.85, and a batch in which 0.8% of the rows (nine of 1,200) are moved seven units along each of the four coordinates, a distance of 14.

import numpy as np


def dist(A, B):
    return np.sqrt(np.maximum(((A[:, None, :] - B[None, :, :]) ** 2).sum(-1), 0))


def mixture(w, n, rng, d=4):
    mu = np.zeros(d)
    mu[0] = 3.0
    return rng.normal(size=(n, d)) + np.where((rng.random(n) < w)[:, None], mu, 0.0)


rng = np.random.default_rng(2024)
n = 1200
train = mixture(0.50, 3000, rng)
self_d = np.sort(dist(train, train), axis=1)[:, 1:11].mean(1)
gate = np.quantile(self_d, 0.999)


def novel_rate(X):
    return float((np.sort(dist(X, train), axis=1)[:, :10].mean(1) > gate).mean())


base = mixture(0.50, n, rng)
far = mixture(0.50, n, rng)
far[:int(0.008 * n)] += 7.0
cases = {"same as training": mixture(0.50, n, rng),
         "mixture weight 0.50 to 0.85": mixture(0.85, n, rng),
         "0.8% of rows from far away": far}

print("gate (99.9th percentile of training neighbor distance): %.3f" % gate)
for name, cur in cases.items():
    print("%-30s %6.2f%% of rows past the gate" % (name, novel_rate(cur) * 100))

gate (99.9th percentile of training neighbor distance): 2.225
same as training                 0.00% of rows past the gate
mixture weight 0.50 to 0.85      0.00% of rows past the gate
0.8% of rows from far away       0.75% of rows past the gate

Only the batch with far-away rows trips the gate: 0.75% of its rows, which is the nine moved rows. The batch whose mixture weight moved has 0.00% past the gate, because shifting weight between two modes the training data already covers puts no row anywhere new.

The batch-level detector is a permutation test on energy distance between the baseline batch and each current batch. The statistic is twice the mean pairwise Euclidean distance between the two batches minus the mean distance within each; perm_p shuffles the pooled rows between the two groups 400 times and reports the share of shuffles whose statistic is at least the observed one, with the observed value counted once.

def energy_from(D, idx, m):
    a, b = idx[:m], idx[m:]
    return (2 * D[np.ix_(a, b)].mean() - D[np.ix_(a, a)].mean() - D[np.ix_(b, b)].mean())


def perm_p(X, Y, reps=400):
    D, m = dist(np.vstack([X, Y]), np.vstack([X, Y])), len(X)
    idx = np.arange(len(D))
    stat = energy_from(D, idx, m)
    rng, hits = np.random.default_rng(3), 0
    for _ in range(reps):
        rng.shuffle(idx)
        hits += energy_from(D, idx, m) >= stat
    return (hits + 1) / (reps + 1)


for name, cur in cases.items():
    print("%-30s population p %.4f" % (name, perm_p(base, cur)))

same as training               population p 0.6185
mixture weight 0.50 to 0.85    population p 0.0025
0.8% of rows from far away     population p 0.0698

The middle batch is a population that moved (p = 0.0025) built entirely out of rows that are individually unremarkable. The bottom batch is nine rows from nowhere, hiding inside a population whose test would not trip a 5% alarm (p = 0.0698). The same two numbers per batch, the population p-value and the share of rows past the gate, point in opposite directions.

The practical reading is that these need separate instruments and separate responses. A population change calls for diagnosis and impact assessment; it need not require retraining. A novel row may be handled by fallback, review, abstention or ordinary processing under a validated risk policy.

Confidence is an imperfect novelty signal

The obvious place to look for the row-level answer is the model’s own output, but confidence is not a universal novelty detector. Lee et al. state the problem in their abstract: softmax classifiers produce highly overconfident posterior distributions even for abnormal samples. It is worth seeing why that is structural rather than a training defect. Three Gaussian classes are fitted with a softmax (multinomial logistic) classifier, and a point is then moved straight up from the center of class 0 by the distance in the first column.

from sklearn.linear_model import LogisticRegression

rng = np.random.default_rng(11)
n = 2000
centers = np.array([[0.0, 2.0], [-2.0, -1.0], [2.0, -1.0]])
X = np.vstack([ctr + rng.normal(size=(n, 2)) for ctr in centers])
y = np.repeat([0, 1, 2], n)
clf = LogisticRegression(max_iter=5000).fit(X, y)

print("in-sample accuracy %.4f" % clf.score(X, y))
print("%14s %16s %14s" % ("distance", "max softmax", "predicted"))
for r in (0, 2, 5, 10, 25, 50, 200):
    pt = np.array([[0.0, 2.0 + r]])
    p = clf.predict_proba(pt)[0]
    print("%14.0f %16.6f %14d" % (r, p.max(), int(p.argmax())))

in-sample accuracy 0.9423
      distance      max softmax      predicted
             0         0.996192              0
             2         0.999988              0
             5         1.000000              0
            10         1.000000              0
            25         1.000000              0
            50         1.000000              0
           200         1.000000              0

Softmax probabilities depend on the differences between all class logits. Along the direction used in this example, the winning linear logit increasingly exceeds every competing logit, so its probability approaches one even as the point moves farther from the observed training clusters. The model has no term anywhere in it for how far this input is from anything I have seen, so it cannot report that.

Ask where the row is instead

Lee et al.’s fix is to fit a Gaussian to the network’s features for each class and score a new row by its smallest Mahalanobis distance to those. That score asks where the row sits rather than what the model thinks of it. The block below continues with the classifier fitted above. mahalanobis_score uses the class means and a pooled within-class covariance, and returns the smallest squared distance over the classes. Each of three out-of-distribution sets is mixed with 1,800 in-distribution rows, and AUROC measures how well a score ranks the out-of-distribution rows above them, for the softmax score (one minus the maximum probability) and for the distance.

from sklearn.metrics import roc_auc_score

mus = np.array([X[y == k].mean(0) for k in range(3)])
shared = sum(np.cov((X[y == k] - mus[k]).T) for k in range(3)) / 3
inv = np.linalg.inv(shared)


def mahalanobis_score(Z):
    d = np.stack([np.einsum("ij,jk,ik->i", Z - m, inv, Z - m) for m in mus])
    return d.min(0)


inlier = np.vstack([ctr + rng.normal(size=(600, 2)) for ctr in centers])
suites = {
    "shifted 6 units, class 0 side": np.array([0.0, 8.0]) + rng.normal(size=(1800, 2)),
    "scattered far field": rng.uniform(-30, 30, (1800, 2)),
    "squeezed into the middle": rng.normal(0, 0.15, (1800, 2)),
}

print("%-32s %18s %18s" % ("out-of-distribution set", "AUROC softmax", "AUROC Mahalanobis"))
for name, ood in suites.items():
    lab = np.concatenate([np.zeros(len(inlier)), np.ones(len(ood))])
    both = np.vstack([inlier, ood])
    soft = 1 - clf.predict_proba(both).max(1)
    print("%-32s %18.4f %18.4f"
          % (name, roc_auc_score(lab, soft), roc_auc_score(lab, mahalanobis_score(both))))

out-of-distribution set               AUROC softmax  AUROC Mahalanobis
shifted 6 units, class 0 side                0.0001             0.9999
scattered far field                          0.0782             0.9933
squeezed into the middle                     0.9901             0.8949

On the first suite, confidence scores 0.0001, which is not a weak detector but an inverted one: those rows are further from the boundary than any training point, so ranking by low confidence puts them at the very bottom of the list. The distance score reads 0.9999 on the same rows.

The third suite is the reason to keep both. Those rows are piled in the gap between the clusters, well inside the training region, where the model is genuinely undecided. Confidence finds them at 0.9901 and distance is the weaker of the two. Low confidence and off-distribution are different conditions with different repairs — neither automatically dictates human review or refusal; select the response from validated error costs and available fallback options. Both scores are computed on the model’s features, which makes them a representation measurement: if the encoder that produces those features is refitted or swapped, the scores change even when the data does not.

Hendrycks and Gimpel’s result stands alongside this rather than against it: maximum softmax probability is a simple baseline that tends to rank correctly classified examples above misclassified and out-of-distribution ones across tasks in vision, language and speech. The first suite is the case a monitor still has to survive, and the worst case for confidence is an input that is far away in a confident direction.

References

• Hendrycks, D., & Gimpel, K. (2017). A baseline for detecting misclassified and out-of-distribution examples in neural networks. International Conference on Learning Representations (ICLR).
• Lee, K., Lee, K., Lee, H., & Shin, J. (2018). A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in Neural Information Processing Systems (NeurIPS), 31.