Probabilistic Forecasts

A probabilistic forecast states a distribution over future values rather than a single number. In the standard reference’s framing, “we express the uncertainty in our forecasts using a probability distribution” that “describes the probability of observing possible future values using the fitted model” — and the familiar single number is derived from it, since “the point forecast is the mean of this distribution.”

So a probabilistic forecast is not an elaboration of a point forecast; a point forecast is a summary of it. Which way round that is matters for how you build and report forecasts.

What keeping the distribution buys

Three things a single number cannot deliver.

• The decision picks its own summary. Stock planning may want a high demand quantile and financial reporting a mean. An SLA for latency may use a high latency quantile, while a lower bound on availability concerns a low availability quantile; the direction depends on the outcome and the commitment. With the distribution retained upstream, each consumer chooses; with a mean written to a table, they all inherit a choice nobody made for them. See quantile forecasts.
• Risk becomes visible. Questions like “how likely is demand above capacity?” are answered by integrating a region of the distribution, and are unanswerable from a point value — see risk forecast and Value-at-Risk.
• Uninformative forecasts announce themselves. A wide distribution is visibly wide. A mean of 500 looks identical whether the interval is [480, 520] or [50, 950].

The authors’ verdict on the alternative is blunt: “point forecasts can be of almost no value without the accompanying prediction intervals.” See point forecasts and prediction intervals; the version that reports nothing but a number is a deterministic forecast.

The normality assumption is usually doing work

Worth knowing before trusting the tails: “most time series models produce normally distributed forecasts — that is, we assume that the distribution of possible future values follows a normal distribution.” That assumption is often reasonable near the center and unreasonable in the tails, which is exactly where risk questions live.

The documented alternative when it fails is resampling: “when a normal distribution for the residuals is an unreasonable assumption, one alternative is to use bootstrapping, which only assumes that the residuals are uncorrelated with constant variance.” The source states a lack of correlation, which does not imply independence. For example, if X is equally likely to be -1, 0, or 1 and Z=X², their covariance is zero but Z depends on X. Ordinary residual resampling needs stronger conditions such as suitably iid innovations; dependent data may need block or model-based resampling. An ACF check alone cannot establish those conditions. See bootstrap and normal distribution.

Scoring it needs different measures

Accuracy metrics for point forecasts do not evaluate a distribution: “the preceding measures all measure point forecast accuracy. When evaluating distributional forecasts, we need to use some other measures.” There are three, and they differ in what they score.

The Winkler score makes a point that corrects a common instinct. For observations inside the interval it “is simply the length of the interval. Thus, low scores are associated with narrow intervals” — but a penalty applies when the observation falls outside, “proportional to how far the observation is outside.” So narrow is rewarded and dishonestly narrow is punished, which is the right shape: an interval is not better for being tighter if the outcomes fall outside it.

Use CRPS when you care about the whole distribution and a quantile score when one specific quantile drives a decision. And in either case check calibration separately from score — count how often outcomes fell inside a stated interval and compare with the level claimed, because a good average score can coexist with systematically overconfident tails.

References:

• Hyndman & Athanasopoulos, Forecasting: Principles and Practice (3rd ed.), §5.5 “Distributional forecasts and prediction intervals”
• Hyndman & Athanasopoulos, Forecasting: Principles and Practice (3rd ed.), §5.9 “Evaluating distributional forecast accuracy”