Epochs

Definition

An epoch is one complete pass of the training dataset through the model during training.

• In each epoch, the model sees all training samples once (though in mini-batches).
• After multiple epochs, the model refines its weights based on repeated exposure to the data.

---

Key Concepts

1. Epoch vs. Batch vs. Iteration

• Batch = a subset of the training data processed at once.
• Iteration = one update step (forward + backward pass on a batch).
• Epoch = one full cycle through the entire dataset (all batches).

Example:

• Dataset = 10,000 samples
• Batch size = 100
• → 100 iterations = 1 epoch

2. Multiple Epochs

• Training usually requires many epochs.
• Each epoch reduces loss (up to a point).

3. Too Few Epochs

• Underfitting → model doesn’t learn enough.

4. Too Many Epochs

• Overfitting → model memorizes training data, performs poorly on test data.

---

How Epochs Work in Training

1. Initialize weights.
2. For epoch = 1,2,…,N:
   • Loop over mini-batches → forward + backward pass → update weights.
   • At the end of epoch → evaluate on validation set.
3. Stop when performance stops improving (early stopping).

---

Example

• Train an image classifier on CIFAR-10.
• Set epochs = 20.
• During training:
  • Epoch 1 → model sees all 50k images once.
  • Epoch 2 → sees them again, adjusts weights further.
  • By epoch 20 → loss stabilized, accuracy ~85%.

---

Practical Notes

• Epoch count is a hyperparameter → must be tuned.
• Typical ranges:
  • Small datasets: 50–200 epochs.
  • Large datasets (with early stopping): 5–30 epochs.
• Early stopping is often used to stop training once validation loss stops improving.

---

Summary
An epoch = one complete pass through the entire training dataset.

• It consists of many iterations (batches).
• More epochs → better learning, but risk of overfitting.
• Training uses techniques like early stopping to find the right number of epochs.