Mean Squared Error (MSE)

Mean squared error is the average of the squared differences between predicted and actual values: with \(e_i = y_i - \hat{y}_i\) over \(n\) observations, \(\text{MSE} = \frac{1}{n}\sum e_i^2\). The scikit-learn documentation describes it as the expected value of the squared error loss, and the Google crash course as the average of the L2 losses over a set of examples. It is the quantity that least-squares regression minimizes during training, and its square root, the root mean squared error, is the quantity usually reported.

Squaring does two jobs. It removes the sign, so that errors in opposite directions do not cancel when averaged. And it changes the weighting: “When the difference between the prediction and label is large, squaring makes the loss even larger. When the difference is small (less than 1), squaring makes the loss even smaller.” That is the whole difference between the squared and absolute families: MAE averages the absolute errors, MSE averages their squares. RMSE belongs to the squared family; it is MSE with a square root taken at the end, which changes the units of the reported number and nothing about which model scores best on a given set of predictions.

A worked example

The MSE is (1 + 0 + 4 + 1) / 4 = 1.5, in squared units of the target. The one error of 2 contributes four of the six units in the sum; the two errors of 1 contribute one each. The mean absolute error of the same predictions is 1.0, and the RMSE is about 1.22.

Why it is minimized, and what that aims at

MSE is a common training loss for regression for a practical reason: it is smooth and differentiable everywhere, so gradient-based fitting works, and for a linear model it has a closed-form minimum. Minimizing it aims the model at the conditional mean; the forecasting reference states that “minimising the RMSE will lead to forecasts of the mean”, and MSE and RMSE have the same minimizer. The crash course frames the choice against MAE as a choice about how the model should treat outliers, which MSE penalizes more heavily.

One result is often quoted about MSE and deserves its conditions. For an estimator of a fixed quantity, the expected squared error decomposes into the variance of the estimator plus the square of its bias, which is why fitting procedures that reduce variance at the cost of some bias can lower expected squared error overall. That is a statement about expected error over repeated samples, not about the sample MSE of one set of predictions; the sample MSE is an estimate of the expected squared loss, and reading a bias-variance split off a single number is not possible.

Where it fits, and where it does not

from sklearn.metrics import mean_squared_error
mean_squared_error([3, 5, 2, 7], [2, 5, 4, 6])
# 1.5

How the squared, absolute, percentage, and scaled families relate, and how the choice among them changes what a forecast is aiming at, is set out under forecast error.

References

• scikit-learn, Metrics and scoring: regression metrics
• Google, Machine Learning Crash Course: Linear regression: Loss
• Rob J. Hyndman and George Athanasopoulos, Forecasting: Principles and Practice (3rd ed.), section 5.8 Evaluating point forecast accuracy