Quantile Level

The quantile level is the probability p used to select a position in the outcome distribution. For \(0<p<1\), define a quantile as \(q_p=\inf\{x:F(x)\ge p\}\). It satisfies \(P(Y<q_p)\le p\le P(Y\le q_p)\). For a continuous distribution the probability below the quantile is p; with point masses or ties, exact equality need not hold.

For a continuous distribution, a calibrated 0.9 quantile has a 10% exceedance probability. For discrete outcomes, it can be lower: if demand is 0 with probability 94% and 1 otherwise, the 0.9 quantile is 0 and its exceedance probability is 6%. The level describes the outcome distribution, not confidence in the estimated quantile.

Choosing it from the costs

The level should come from the decision, not from convention. If running short costs \(c_u\) per unit and holding excess costs \(c_o\) per unit, the cost-minimizing level is

\[p = \frac{c_u}{c_u + c_o}\]

so equal costs point at the median, a stockout three times as painful points at 0.75, and one ten times as painful at about 0.91. (The ratio is the author’s derivation; the definition of the quantile forecast above is the source’s.) Deriving \(p\) from the costs is the whole discipline here — reaching for 0.95 because it is familiar imports a cost ratio of 19 to 1 that nobody chose. See quantile forecasts.

Levels also assemble into intervals. “Prediction intervals are usually constructed from quantiles by setting \(\ell_{\alpha,t} = f_{\alpha/2,t}\) and \(u_{\alpha,t} = f_{1-\alpha/2,t}\)” — so a 90% interval is the 0.05 and 0.95 levels, and the level of an interval bound is not the same number as the coverage of the interval. See prediction intervals.

Three things the level does not do

• It does not make the number a forecast of the mean. Judging a 0.95 level by how close it came to the actual value scores it as if it were a mean. Use a matching quantile score and calibration checks that distinguish outcomes below, equal to, and above the forecast.
• It does not add across items. In general, adding component quantiles does not give the total’s quantile; the error can go either way and depends on the joint distribution. Two independent demands, each 0 with probability 94% and 1 otherwise, each have a 90th percentile of 0, but their sum has a 90th percentile of 1. Some forms of perfect positive dependence do permit addition. Whether stock can be pooled across products is a separate operational question.
• It does not pick its own scoring function. A quantile at level \(p\) must be scored with the matching pinball loss; the source notes that only at \(p = 0.5\) does that coincide with absolute error. Score a 0.9 level by MAE and you reward pulling it toward the middle.

The verification that matters is calibration. Hold out periods, count how often the outcome fell below each level you publish, and compare with the level itself. A “0.9” that is exceeded a third of the time is a measurement you can act on — and the label alone is not evidence about the behavior. Interpret these frequencies using ties, sampling uncertainty, and temporal dependence; equality to p is not required for a distribution with point masses. For the whole distribution rather than one level, see CRPS and probabilistic forecasts.

Further reading

• Hyndman & Athanasopoulos, Forecasting: Principles and Practice (3rd ed.), §5.9 “Evaluating distributional forecast accuracy”
• §5.8 “Evaluating point forecast accuracy”
• §5.5 “Distributional forecasts and prediction intervals”