Interpreting Analytical Results
The question is not whether a result is significant. It is what the sample represents, how wide the uncertainty is, and whether the difference matters.
-
-
Gaussian Processes (GPs)
Standard deviation of 0.1123 where observations were dense, 0.9620 in the gap between them, and 1.0046 beyond all the data.
-
Bayesian Neural Networks (BNNs)
Bayesian neural networks average predictions over a weight posterior. Prior, likelihood, inference approximation, and observation noise shape their uncertainty.
-
Deterministic forecasts
Reported without an interval, a forecast carries no way of telling how accurate it is — which is why a single line reads as certainty.
-
Prediction Intervals (PI)
Interval width depends on the model and horizon. Coverage reflects the uncertainty represented in the model, not every possible change in the world.
-
Underconfident
Probabilities too close to 0.5, intervals wider than the evidence requires. The mirror of overconfidence, and repairable the same way.
-
Overconfident
Probabilities too close to 0 and 1, intervals too narrow. A monotone sharpening can preserve accuracy at 0.5 and AUC while worsening probability estimates.
-
Deep Ensembles
Deep ensembles average separately trained networks. Error correlation limits gains, and member disagreement captures only part of predictive uncertainty.
-
Uncertainty Estimation and Model Calibration
A softmax output of 0.99 is not a 99% chance of being right. Here is how far apart those two things are.
-
Gaussian Processes and Bayesian Regression
Every other model returns a number. This one returns a distribution.