Sliding Window (Rolling Window) Cross-Validation
After a regime change, the sliding window recovers to an MAE of 0.43 while the expanding window stalls at 2.79.
-
-
Macro Shifts
Eight features, not one of them past its noise floor, and their mean correlation has gone from +0.0016 to +0.1985.
-
Concept Drift
The inputs look the same and the model is unchanged, but what those inputs predict has moved. Only outcomes can show it, and outcomes arrive late.
-
Data Drift
Distribution changes can alter aggregate error without changing the input-to-outcome relationship. How to interpret drift and choose a response.
-
Drift Guardrails
An intercept change is not pure label shift. The example separates the assumptions behind log-odds correction from an AUC-drop gate.
-
Dataset Shift
A new population can alter model quality, but distribution change and performance loss are different claims. Diagnose the shift before choosing a correction.
-
Monitoring, Drift, and Retraining
The alarm and the harm are different things.