Hyperparameter Tuning for Deep Learning
Tuning is a search problem with a budget. Spend it on the parameters that matter.
-
-
Preventing Overfitting in Deep Learning
A practical framework for recognizing overfitting and choosing the right regularization method.
-
Normalization Layers in Deep Learning
Every normalization layer answers the same question differently: which axes do we average over?
-
Learning Rate Schedules and Warmup
The learning rate is the single most important hyperparameter, and a constant value is almost never the right choice.
-
Deep Learning Optimization Algorithms
Understand what popular deep-learning optimizers remember, how they update parameters, and when to tune the learning rate.
-
Gradient Checking for Neural Networks
A wrong gradient produces a model that trains slowly instead of crashing. Here is how to catch it.
-
Weight Initialization and Gradient Flow
Initialization is not a detail. It decides whether a deep network trains at all.
-
Deep Learning Loss Function Design
The loss function is the only place where you tell a network what “better” means. Choose it deliberately.
-
Softmax and Multiclass Classification Explained
Extend binary logistic regression to many classes with softmax, cross-entropy, and a numerically stable NumPy implementation.