Mean Integrated Squared Error

Mean integrated squared error measures how far an estimated curve sits from the true one. Take the squared difference at every point, integrate over the range to get one number per run, then average across hypothetical repeated runs. Squared because sign should not matter, integrated because the whole curve matters, mean because the estimate depends on the particular sample you drew.

It is the standard criterion for judging density estimates and histograms, where the object being estimated is a function rather than a single number. A criterion based on one point would let an estimator be excellent at that point and nonsense elsewhere.

The value of MISE is that it decomposes. Part of the error comes from the estimator being systematically off — a histogram bar is flat across an interval where the truth is changing — and part from the estimator chasing randomness in a finite sample. Wider bins reduce the second and increase the first; narrower bins do the reverse. The best setting is wherever the sum is smallest, which is a property of the data rather than a universal constant.

The obvious problem is that MISE is defined against a truth you do not have. If you did, you would not be estimating it. So MISE is rarely computed; it is manipulated. Terms that do not depend on the setting being chosen shift the curve vertically without moving its minimum, so they can be dropped, and with a distributional assumption the rest can sometimes be rewritten in terms of observable quantities.

What survives is a cost function whose expectation, under the method’s assumptions, differs from MISE by a term that does not depend on the setting. That expected criterion has the same minimum even if its values can be negative. In practice we estimate the cost from the observed sample: its minimizer can vary from sample to sample and need not equal the minimum of the unknown MISE. The trick supplies an estimate of the best setting, rather than a guarantee of an identical choice for every sample.

See it in use in Choosing Histogram Bins for Time-Indexed Events.

References

• Shimazaki & Shinomoto, bin size selection method