RMSLE (Root Mean Squared Logarithmic Error)

RMSLE takes the logarithm of actual and forecast before comparing them:

\[ \text{RMSLE} = \sqrt{\frac{1}{N}\sum_{i=1}^{N}\big(\log(\hat{y}_i + 1) - \log(y_i + 1)\big)^2} \]

Because a difference of logarithms is the logarithm of a ratio, this scores proportional error rather than absolute error. The \(+1\) keeps it defined when a value is zero.

What it actually measures

import numpy as np

squared_log_error = lambda y, p: (np.log(p + 1) - np.log(y + 1)) ** 2
print(f"{'actual':>9}{'forecast':>10}{'raw miss':>11}{'squared log error':>20}")
for yv, pv in ((100, 90), (1000, 900), (10, 9)):
    print(f"{yv:>9}{pv:>10}{yv - pv:>11}{squared_log_error(yv, pv):>20.6f}")
"""But the logarithm is not symmetric, and the +1 is not neutral."""

print(f"\n{'actual':>9}{'forecast':>10}{'direction':>13}{'squared log error':>20}")
for pv in (200, 50):
    print(f"{100:>9}{pv:>10}{'double' if pv > 100 else 'half':>13}{squared_log_error(100, pv):>20.6f}")
for pv in (110, 90):
    print(f"{100:>9}{pv:>10}{'+10%' if pv > 100 else '-10%':>13}{squared_log_error(100, pv):>20.6f}")
print(f"  doubling and halving cost almost the same ({squared_log_error(100, 200) / squared_log_error(100, 50):.4f}),")
print(f"  but under-forecasting by 10% costs "
      f"{squared_log_error(100, 90) / squared_log_error(100, 110):.4f} times an over-forecast of 10%")

print(f"{'actual':>9}{'forecast':>10}{'ratio':>9}{'with +1':>12}{'pure log ratio':>17}")
for yv, pv in ((1, 2), (10, 20), (100, 200), (10_000, 20_000)):
    print(f"{yv:>9}{pv:>10}{pv / yv:>9.1f}{squared_log_error(yv, pv):>12.6f}"
          f"{np.log(pv / yv) ** 2:>17.6f}")
#    actual  forecast   raw miss   squared log error
#       100        90         10            0.010870
#      1000       900        100            0.011077
#        10         9          1            0.009084
#
#    actual  forecast    direction   squared log error
#       100       200       double            0.473598
#       100        50         half            0.466892
#       100       110         +10%            0.008913
#       100        90         -10%            0.010870
#   doubling and halving cost almost the same (1.0144),
#   but under-forecasting by 10% costs 1.2196 times an over-forecast of 10%
#    actual  forecast    ratio     with +1   pure log ratio
#         1         2      2.0    0.164402         0.480453
#        10        20      2.0    0.418127         0.480453
#       100       200      2.0    0.473598         0.480453
#     10000     20000      2.0    0.480384         0.480453

The tables show squared log-error contributions to MSLE, before averaging and taking the square root to obtain RMSLE. The first table illustrates proportional scaling. A 10% miss costs 0.010870 when the actual is 100, 0.011077 when it is 1,000, and 0.009084 when it is 10. The raw errors differ by a factor of a hundred and the scores barely move. On a target spanning several orders of magnitude — sales across a catalog, counts across regions — that is exactly the behavior RMSE lacks, since RMSE would let the largest items decide the whole number.

The second table is the part that is usually left out. Doubling and halving cost nearly the same (ratio 1.0144), which is the ratio-symmetry the logarithm buys. But at 10% the two directions are not equal: the squared log-error contribution of under-forecasting is 1.2196 times that of over-forecasting. This is not the ratio of the two single-observation RMSLE scores.

The asymmetry is real and it has a direction: RMSLE leans against under-prediction. For demand forecasting that often matches the business, where a stockout costs more than a little excess. The point is that it is a choice being made by the metric, and it should be a choice you agreed to rather than one you inherited from a competition leaderboard.

The +1 is not neutral

The last table is the detail that quietly breaks the scale-free claim. A forecast that doubles the actual contributes 0.480384 to MSLE when the actual is 10,000, 0.473598 at 100, 0.418127 at 10 — and only 0.164402 at 1. The squared pure log-ratio is 0.480453 in every row. For a single observation, RMSLE is the absolute log difference: approximately 0.6931 for actual 10,000 and forecast 20,000, and 0.4055 for actual 1 and forecast 2.

So at small counts the \(+1\) dominates and the same proportional error is scored far more leniently. If your target is mostly small integers — daily units for slow-moving items, rare event counts — RMSLE is not measuring what its description promises, and the offset is doing most of the work.

Two related cautions follow. The \(+1\) is a convention, not a law; some implementations use a different offset or require strictly positive inputs, so an RMSLE from one library is not automatically comparable to another's. And RMSLE is undefined for negative values, which rules it out for anything that can go below zero.

Where it belongs

RMSLE suits targets that are non-negative, span orders of magnitude, and where proportional error is the thing that matters. That combination is common enough — demand, traffic, prices — that it has become a default in forecasting competitions.

It is a poor choice when the target is near zero for most rows, when errors have direct linear costs in units, or when the audience needs a number in the target's own units. In those cases MAE or WAPE say something a planner can act on, and RMSLE says something about ratios that has to be translated first.

References

• NumPy documentation, numpy.log. numpy.org