Concept Drift

1) Definition

• Concept drift = when the relationship between input features and the target variable ($P(Y|X)$) changes over time.
• Unlike covariate drift (inputs change) or label drift (class balance changes), here the “meaning” of the prediction changes.

Example: The same features now map to different outcomes.

---

2) Example Scenarios

• Spam detection
  • Past: “free gift” → usually spam.
  • Present: “free gift” → common in legitimate promotions.
• Credit risk
  • Past: high income → low risk.
  • After a recession: high income applicants may still default → relationship shifts.
• Healthcare
  • Past: certain symptoms → mild illness.
  • During a pandemic: same symptoms → severe disease risk.

---

3) Causes of Concept Drift

• External changes: economy, laws, pandemics, cultural shifts.
• Adversarial adaptation: fraudsters/spammers evolve tactics.
• Population changes: new types of users enter the system.

---

4) Types of Concept Drift

1. Sudden drift
   • Concept changes quickly.
   • Example: New regulation → loan approval rules shift overnight.
2. Gradual drift
   • Transition over time.
   • Example: Customer shopping preferences slowly change.
3. Recurring drift
   • Concept reappears.
   • Example: Winter clothing demand spikes every year.

---

5) Detection Methods

• Performance monitoring
  • Drop in accuracy, AUC, calibration → indicates concept drift.
• Statistical tests
  • Compare conditional distributions $P(Y|X)$ over time.
• Drift detection algorithms
  • DDM (Drift Detection Method)
  • EDDM (Early Drift Detection Method)
  • ADWIN (Adaptive Windowing)

---

6) Handling Concept Drift

• Frequent retraining: update model with fresh data.
• Online learning: continuously adapt weights as new data arrives.
• Ensembles: keep a pool of models trained on different time windows.
• Hybrid approach: retrain only when drift is detected (trigger-based).

---

7) Example

Fraud detection system:

• Training: unusual IP address → likely fraud.
• Attackers adapt: use normal IPs → the same feature no longer predicts fraud.
• Model AUC drops → drift guardrail triggered → retrain model on new fraud patterns.

---

Summary

• Concept drift = change in $P(Y|X)$, i.e., features → label relationship.
• Harder to detect than covariate/label drift.
• Can be sudden, gradual, recurring.
• Detection: performance monitoring, drift detectors.
• Mitigation: retraining, online learning, ensembles.