Definition
An epoch is one complete pass of the training dataset through the model during training.
- In each epoch, the model sees all training samples once (though in mini-batches).
- After multiple epochs, the model refines its weights based on repeated exposure to the data.
Key Concepts
- Epoch vs. Batch vs. Iteration
- Batch = a subset of the training data processed at once.
- Iteration = one update step (forward + backward pass on a batch).
- Epoch = one full cycle through the entire dataset (all batches).
Example:
- Dataset = 10,000 samples
- Batch size = 100
- → 100 iterations = 1 epoch
- Multiple Epochs
- Training usually requires many epochs.
- Each epoch reduces loss (up to a point).
- Too Few Epochs
- Underfitting → model doesn’t learn enough.
- Too Many Epochs
- Overfitting → model memorizes training data, performs poorly on test data.
How Epochs Work in Training
- Initialize weights.
- For epoch = 1,2,…,N:
- Loop over mini-batches → forward + backward pass → update weights.
- At the end of epoch → evaluate on validation set.
- Stop when performance stops improving (early stopping).
Example
- Train an image classifier on CIFAR-10.
- Set epochs = 20.
- During training:
- Epoch 1 → model sees all 50k images once.
- Epoch 2 → sees them again, adjusts weights further.
- By epoch 20 → loss stabilized, accuracy ~85%.
Practical Notes
- Epoch count is a hyperparameter → must be tuned.
- Typical ranges:
- Small datasets: 50–200 epochs.
- Large datasets (with early stopping): 5–30 epochs.
- Early stopping is often used to stop training once validation loss stops improving.
Summary
An epoch = one complete pass through the entire training dataset.
- It consists of many iterations (batches).
- More epochs → better learning, but risk of overfitting.
- Training uses techniques like early stopping to find the right number of epochs.