Diagnosing and Preventing Overfitting in Deep Learning

A model can improve its fit to the training examples while becoming less useful on new ones. Overfitting is this excessive adaptation to the training sample relative to the intended prediction task. A training–validation gap is one sign to investigate, but neither perfect training accuracy nor a large gap alone establishes the cause.

It is tempting to respond by adding regularization immediately. However, that response can make the model worse when the real problem is underfitting, weak features, unsuccessful optimization, or an evaluation set that does not represent production. A reliable workflow begins with diagnosis and chooses a remedy only after the pattern of errors is understood.

Diagnose bias and variance

Use the same metric and comparable evaluation conditions for training and validation (also called development or dev) data. High training error relative to a credible reference suggests room to improve the fit; a large validation gap suggests that the fit transfers poorly. Human performance is a benchmark, not Bayes error: Bayes error is the lowest population classification error attainable with the available inputs and is usually unknown. Humans may use different information, and models can outperform them. These comparisons are diagnostic clues, not estimates of the formal bias and variance components. Variance concerns how predictions change across different training samples; a single train–dev gap does not measure that variation directly.

How to decide which improvement to attempt next is the subject of A Systematic Strategy for Improving Machine Learning Models.

Read the learning curves before choosing a remedy

Imagine that a cat classifier reaches 99% accuracy on training images and 78% on validation images. That is a 21-percentage-point gap. It can fit the training labels, but these two numbers do not show whether it relies on backgrounds, camera patterns, or another feature. Check source-specific results or controlled changes to images to investigate those hypotheses. A larger model might change generalization, but fitting the training set more accurately is not sufficient evidence that it will close this gap.

A second model reaches 72% on training data and 70% on validation data. The two-point gap says little about whether 70% is acceptable: compare it with the class-balance baseline, task requirements, and credible reference models. If those references are substantially better, investigate weak features, insufficient training, unsuitable capacity, label quality, or excessive regularization. A small gap alone does not establish a good fit.

Plot training and validation loss against epochs, where an epoch is one pass through the training set. A sustained validation-loss rise while training loss falls is evidence for selecting an earlier checkpoint on that loss, after checking noise and evaluation consistency. It does not identify memorization or the particular features responsible. If both curves stay high relative to a useful baseline, inspect input scaling, learning rate, optimization, and model capacity.

Compare data losses under the same evaluation procedure: disable dropout, use the same fixed preprocessing, and exclude the training-only regularization penalty from both curves. You can log the stochastic, augmented training objective separately. Otherwise validation loss can be lower simply because training examples were made harder. Track the deployment metric too; a rise in validation loss need not imply a fall in accuracy, and not every useful training loss is smooth.

Why a clean validation set matters

Validation guides architecture, regularization strength, and training duration. Overlapping users, images, or sessions can make validation optimistic when deployment requires generalization to new ones. Choose the splitting unit to match that requirement: predicting future behavior of known users and predicting for entirely new users are different evaluation tasks.

For example, a user-level prediction problem may require a user-level split so that one person’s records do not appear on both sides. A forecasting problem normally requires a chronological split because production predictions concern the future, not a random sample of the past. Image collections may need grouping by subject, source, or capture session to prevent near-duplicates from leaking across the boundary.

Repeatedly tuning decisions against the same validation set creates a subtler form of overfitting. Preserve a final test set that does not influence model selection. The validation set guides development; the test set estimates performance after the important choices have been made.

After auditing evaluation, compare interventions under a fixed protocol. More independent, representative training cases can reduce sampling variability; duplicate records do not provide the same information, and adding shifted or mislabeled data need not help. The following techniques modify different parts of training.

L2 regularization

For \(m\) training examples, one convention is \(J=J_{\rm data}+\frac{\lambda}{2m}\sum_\ell\|W^{[\ell]}\|_F^2\), where \(J_{\rm data}\) is mean data loss, \(\lambda\ge0\) controls the penalty, and the squared Frobenius norm sums the squares of all entries in each layer’s weight matrix. Its derivative adds \(\lambda W/m\). Here \(m\) is the full training-set size; using minibatch means for the data term does not require replacing this denominator by the minibatch size. Other implementations omit \(1/m\) from the penalty, so the numeric strength is convention-dependent.

For plain gradient descent with learning rate \(\eta\), the update is \(W\leftarrow(1-\eta\lambda/m)W-\eta\nabla_W J_{\rm data}\). The penalty component shrinks weights when \(0<\eta\lambda/m<1\), but the full update can still increase a weight. With \(w=2,m=10,\lambda=1,\eta=0.1\) and zero data gradient, the new value is 1.98. Biases are often excluded so that an overall offset is not discouraged; this is a modeling convention, not a universal claim that penalizing biases is useless.

L2 penalizes coefficient size but does not by itself guarantee a smooth or robust network. Tune its strength against validation performance and log data loss, penalty, and total objective separately. For adaptive optimizers, adding an L2 term to the gradient is not equivalent to decoupled weight decay: AdamW applies decay separately from its gradient-based update.

Dropout

Elementwise dropout temporarily sets selected activations to zero during training. For a fixed activation \(A\), inverted dropout uses an independent mask \(D\sim\operatorname{Bernoulli}(p)\) and returns \(A_{\rm drop}=DA/p\), where \(0<p\le1\) is the keep probability. Each mask entry is 1 with probability \(p\). Thus \(\mathbb E[A_{\rm drop}\mid A]=A\); for \(A=4,p=0.5\), the output is 0 or 8 with equal probability, averaging to 4. The variance changes to \(A^2(1-p)/p\), so matching the mean does not match the distribution.

For this sampled computation, the backward pass multiplies the incoming gradient by the same \(D/p\). Resampling the mask would differentiate a different computation. Standard validation and inference disable dropout and use the full activations. Later nonlinear layers mean this deterministic prediction need not equal the average prediction over dropout masks. Training across sampled masks can encourage predictive paths that work when some activations are absent, reducing reliance on a fixed combination of units. This can help generalization, but it is not guaranteed. Tune the keep probability; strong dropout can make learning harder, especially when the model already fits poorly.

import numpy as np
rng = np.random.default_rng(0)
A = np.arange(1.0, 7.0).reshape(2, 3)
keep_prob = 0.5
D = rng.random(A.shape) < keep_prob
A_drop = A * D / keep_prob
upstream = np.ones_like(A_drop)
dA = upstream * D / keep_prob
print("mask", D.astype(int), sep="\n")
print("forward", A_drop, sep="\n")
print("backward", dA, sep="\n")
# mask
# [[0 1 1]
#  [1 0 0]]
# forward
# [[0. 4. 6.]
#  [8. 0. 0.]]
# backward
# [[0. 2. 2.]
#  [2. 0. 0.]]

The incoming gradient is all ones here, as it would be for the sum of the dropped activations. Removed entries receive zero gradient; retained entries receive 2. This example keeps the original activation array separate from its dropped version. Library naming differs: PyTorch’s Dropout(p=...) specifies the drop probability, which is 1 - keep_prob.

Data augmentation

Data augmentation varies the inputs seen during training. For ordinary classification, an unchanged target must remain correct after the transformation. Cropping can remove the object, and color changes can erase a class-defining cue. For localization or segmentation, transform boxes or masks along with the image; the target changes consistently rather than remaining identical. Augmented versions are correlated with their source and do not replace independent observations.

Choose transformations that express task-relevant invariances or variations the deployed model should handle. Generate them after splitting the original examples so related versions cannot leak across partitions. Use a fixed validation and test procedure. Test-time augmentation is also possible, but it is part of the prediction rule: choose its transformations and aggregation on validation data, then apply that same rule at test and deployment.

Early stopping

Early stopping monitors a validation metric and saves a checkpoint whenever that metric reaches a new best value. Training stops after a chosen patience period without meaningful improvement. The final model should restore the best checkpoint rather than keep the weights from the last epoch, which may already lie beyond the point where validation performance began to decline.

For validation losses 0.50, 0.40, 0.42, 0.39, 0.41, 0.43, suppose any strict decrease counts as improvement and patience is two consecutive unsuccessful checks. The best checkpoint becomes epoch 4 at 0.39; epochs 5 and 6 fail to improve it, so training stops after epoch 6 and restores epoch 4. A nonzero minimum improvement changes which decreases reset patience, so specify the exact rule.

Record the monitored metric, its direction, minimum improvement, checking frequency, and patience. Choose the metric for the intended use; it need not be loss. Patience handles temporary fluctuations but does not guarantee stopping in finite time if improvements keep arriving, so also set a maximum training budget. The selected validation score has influenced checkpoint selection and is not an independent final estimate.

Choose interventions from evidence

• When training performance trails a credible reference, inspect optimization, labels, features, capacity, and existing regularization.
• Try additional independent, representative data when sampling variability is a plausible problem.
• Compare L2 strengths when penalizing coefficient size is appropriate; do not infer input robustness from the penalty alone.
• Test dropout at selected layers and tune keep probability; a train–validation gap does not diagnose co-adaptation.
• Use augmentation with valid unchanged labels or consistently transformed targets.
• Use validation for early stopping and reserve an untouched test set for final evaluation.

Combine techniques deliberately

Regularization methods are not interchangeable switches. L2 changes the objective by discouraging large weights. Dropout changes the training computation by sampling subnetworks. Augmentation changes the observed training distribution. Early stopping limits how long optimization follows that distribution. Combining them can be effective, but it can also make training unnecessarily difficult. Introduce one change at a time, record the baseline, and compare results under the same split and evaluation procedure.

Compare candidate methods on validation data using a fixed metric and evaluation procedure. Repeat promising comparisons across seeds, and use group- or time-respecting splits where required; a small improvement in one run may reflect randomness. After selecting the configuration, evaluate it on the reserved test data. Regularization cannot repair a misleading split or guarantee performance under a new distribution.

A practical fine-tuning and augmentation recipe is in Image Classification in Practice: Transfer Learning and Augmentation.

Robustness to deliberate perturbation is covered in Adversarial Robustness: Attacks, Defenses, and the Robustness Trade-off.

Exercises

1. Investigate from three numbers. These are classification error rates, not accuracies. For each row, give a plausible explanation, one alternative, and the first check or experiment you would try: (a) train 1%, dev 12%; (b) train 15%, dev 16%, human 2%; (c) train 15%, dev 30%, human 14%; (d) train 0.5%, dev 0.6%, test 8%. Assume the reported metrics use the same definition, but do not assume matching populations or precise estimates from large samples.

You should get: hypotheses and checks; these rates alone do not establish four unique causes.

2. Dropout scaling. Implement inverted dropout with \(p=0.5\) and compare the sampled activation mean with its conditional expectation. Then remove the division by \(p\) and measure how the activation scale changes between training and inference.

You should get: approximately matched means with the correction and a mean near half the original without it.

import numpy as np
rng = np.random.default_rng(0)
A = rng.uniform(1, 2, size=(1000, 1000))
keep = 0.5
D = (rng.random(A.shape) < keep)
print("no dropout      ", round(A.mean(), 4))
print("inverted dropout", round(((A*D)/keep).mean(), 4))
print("plain dropout   ", round((A*D).mean(), 4))
# no dropout       1.5002
# inverted dropout 1.4987
# plain dropout    0.7493

3. Augmentation depends on the target. For horizontal flips of (a) pet photos, (b) handwritten digits, (c) street signs, and (d) chest X-rays, state a task for which unchanged labels might remain valid and a condition that would make the transformation unsuitable. Consider both target correctness and whether the transformed input is plausible for the task.

You should get: task-specific conditions, not a universal verdict for each image category.

References

• Krogh and Hertz (1991). A Simple Weight Decay Can Improve Generalization. NIPS.
• Prechelt (2012). Early Stopping – But When? Neural Networks: Tricks of the Trade.
• Srivastava, Hinton, Krizhevsky, Sutskever, and Salakhutdinov (2014). Dropout: A Simple Way to Prevent Neural Networks from Overfitting. JMLR.
• Zhang, Bengio, Hardt, Recht, and Vinyals (2017). Understanding Deep Learning Requires Rethinking Generalization. ICLR.