Reweighting
The reweighted and plain models rank almost identically at Spearman 0.999959. What reweighting cost was calibration: 0.0009 to 0.2739.
-
-
Dataset Shift
A new population can alter model quality, but distribution change and performance loss are different claims. Diagnose the shift before choosing a correction.
-
Covariate Drift (a.k.a. Covariate Shift)
New kinds of cases can arrive while every rule the model learned still holds. Accuracy can change anyway, because it is an average over a different mix.