Model evaluation K folds and Leave one out

What is the purpose of cross fold validation?

Cross-validation is used to evaluate how well a classification model generalizes to data

How does 10 fold cross validation work?

• Data Division: The dataset is randomly divided into 10 equal parts (folds)

• Stratification: Each fold maintains the same class distribution as the original dataset

• Iterative Testing: For each iteration (10 total):

  One fold is held out as the test set (10% of data)

  Remaining 9 folds (90% of data) are used as the training set. Each fold will have its turn to be used as a test set. the rest will be the training set.

• Final Accuracy: The overall accuracy is calculated as the mean accuracy across all 10 iterations

what are the advantages and disadvantages of 10 fold cross validation?

advantages

• Reduces effects of uneven representation in training/test sets

• Each fold is used exactly once for testing

• Provides statistically robust accuracy estimates

disadvantages

Computationally expensive (train the model 10 times)

how does Leave-one-out Cross-Validation work?

In LOOCV, you:

1. Take just ONE example out as your test set

2. Use ALL other examples as your training set

3. Repeat this process for EVERY example in your dataset

What are the advantages and disadvantages of Leave-one-out Cross-Validation?

Advantages

• Uses maximum possible data for training in each iteration

• Every example gets tested exactly once

Disadvantages

• Extremely computationally expensive for large datasets

• For example, with 5000 examples, you'd need to train 5000 different models!

• Not practical for large datasets or complex models that take long to train