📌 Definition
Cross-validation is a resampling technique used to evaluate how well a model generalizes to unseen data by splitting the dataset into multiple training and validation subsets instead of relying on just one fixed split.
⚙️ How It Works
- Dataset is split into k folds (equal parts).
- Model is trained on k-1 folds and validated on the remaining fold.
- This process is repeated k times, each time using a different fold as the validation set.
- The results (e.g., accuracy, RMSE) are averaged to give a more robust performance estimate.
📊 Example: k-Fold Cross-Validation
- Suppose we use 5-Fold CV:
- Data is split into 5 parts.
- Train on folds 1–4, validate on fold 5.
- Train on folds 1–3,5, validate on fold 4.
- … repeat until every fold has been validation once.
- Take the average accuracy across all 5 runs.
🔑 Types of Cross-Validation
- k-Fold Cross-Validation (most common)
- Splits data into k equal folds.
- Stratified k-Fold CV
- Ensures each fold has the same class distribution (useful for imbalanced datasets).
- Leave-One-Out CV (LOOCV)
- Each data point acts as a test set once (very computationally expensive).
- Repeated k-Fold CV
- Repeats k-fold multiple times with different random splits to reduce variance.
- Time-Series CV (or Rolling Window)
- For sequential data (stock prices, weather), training always happens on past data, validating on future.
✅ Advantages
- Provides a more reliable estimate of model performance than a single train-validation split.
- Makes better use of limited data (since every point gets used for both training and validation).
⚠️ Disadvantages
- Computationally expensive (model is trained multiple times).
- Can be slow for very large datasets or complex models.
🔄 Comparison with Train/Validation/Test Split
| Method | Validation Process | Reliability | Computation |
|---|---|---|---|
| Simple Split | One fixed validation set | Depends on split | Fast |
| Cross-Validation | Multiple rotating folds | More robust, less variance | Slower |
💡 Analogy:
Think of a student preparing for exams. Instead of just taking one practice test, they take 5 practice tests, each with different questions. Averaging their performance gives a better idea of how ready they are.