Definition

Cross-validation is a model evaluation technique that splits data into multiple subsets (folds) to ensure that performance estimates are robust and not overly dependent on a single train/test split.

  • Goal: test how well a model generalizes to unseen data.
  • Helps with model selection, hyperparameter tuning, and overfitting prevention.

How It Works

  1. Split dataset into k folds (subsets).
  2. For each fold:
    • Train model on k–1 folds.
    • Test model on the remaining fold.
  3. Repeat k times (each fold is used as test once).
  4. Average performance across folds → final estimate.

Common Types of Cross-Validation

  1. k-Fold Cross-Validation
    • Data split into k folds (e.g., k=5 or 10).
    • Balanced trade-off between bias & variance.
  2. Stratified k-Fold CV
    • Keeps class distribution consistent across folds (important for imbalanced classification).
  3. Leave-One-Out Cross-Validation (LOOCV)
    • Each fold has exactly 1 sample.
    • Very accurate but computationally expensive.
  4. Time-Series Cross-Validation (Rolling/Expanding Window)
    • For time-dependent data (can’t shuffle).
    • Train on past → validate on future.
  5. Nested Cross-Validation
    • Outer loop: model evaluation.
    • Inner loop: hyperparameter tuning.
    • Prevents overfitting during model selection.

Why It’s Important

  • Reduces variance from one random train/test split.
  • Ensures robust model evaluation.
  • Helps tune hyperparameters (via grid/random/Bayesian search with CV).

Example

Dataset = 1,000 samples, 5-fold CV:

  • Fold 1: Train on 800, validate on 200.
  • Fold 2: Train on 800, validate on 200.
  • … repeat until all folds used.
  • Final score = mean accuracy across 5 folds.

Pros & Cons

Pros

  • More reliable estimate of model performance.
  • Reduces overfitting to one test set.
  • Useful for hyperparameter tuning.

Cons

  • Computationally expensive (train model k times).
  • Must choose fold type carefully (esp. time-series).

Summary
Cross-validation = evaluating a model by splitting data into multiple folds and rotating train/test sets.

  • Types: k-fold, stratified, LOOCV, time-series, nested.
  • Goal: robust, unbiased estimate of generalization performance.