Epochs
An epoch counts data exposure. Batch size, accumulation, sampling, and validation determine what that exposure means for updates and useful learning.
-
-
Learning Rate Schedules and Warmup
The learning rate is the single most important hyperparameter, and a constant value is almost never the right choice.
-
Deep Learning Optimization Algorithms
Understand what popular deep-learning optimizers remember, how they update parameters, and when to tune the learning rate.