Loss Functions
Twenty-five contaminating values pulled the squared-error fit from 99.73 to 142.60, while absolute error stayed at 100.02 and Huber at 100.21.
-
-
Kullback-Leibler Divergence
An optimal code for P costs 1 bit; a code for Q costs 1.736966 on samples from P. The 0.736966 gap is the KL.
-
Log Loss
The negative log-likelihood, used as a metric. Strictly proper, unbounded, and sensitive enough to a single confident error to need watching.
-
Deep Learning Loss Function Design
The loss function is the only place where you tell a network what “better” means. Choose it deliberately.
-
Softmax and Multiclass Classification Explained
Extend binary logistic regression to many classes with softmax, cross-entropy, and a numerically stable NumPy implementation.
-
Logistic Regression for Binary Classification
A practical guide to logistic regression, from probability and cross-entropy to vectorized gradients and a complete NumPy implementation.
-
Probability and Information Theory
Every loss function in this series is a probability statement in disguise. Here is the vocabulary.