Question

Consider that you are training a classifier for a 10-class classification problem. Each I/P is represented as a 512 dimensional vector. There are 1000 samples out of which first 100 will be used for testing. Let Leave-one-out-Cross-Validation (LOOCV) be used for selection of the classifier model before testing. Which of the following option is the correct option for no. of validation split that will be generated?

• A. 10
• B. 512
• C. 1000
• D. 900

Hint:

Remember the hierarchy of data usage in machine learning: The full dataset is split into training and testing sets. Cross-validation techniques like K-Fold or LOOCV are applied *only* on the training set to tune hyperparameters or select a model. The test set is the final, unseen data used to report the model's performance.

Answer 1

The Correct Option is D

Step 1: Understanding the Question:
The question asks for the number of validation splits created when using Leave-one-out-Cross-Validation (LOOCV). It is specified that LOOCV is used for model selection *before* the final testing phase. This means LOOCV is applied only to the training dataset.
Step 2: Key Concepts:
- Train-Test Split: The data is first split into a training set and a testing set. The testing set is kept aside and is only used for the final evaluation of the model.
- Leave-one-out-Cross-Validation (LOOCV): This is a specific type of cross-validation where the number of "folds" or "splits" is equal to the number of samples in the dataset being used. For a dataset of size N, LOOCV involves N iterations. In each iteration, one sample is held out for validation, and the model is trained on the remaining N-1 samples.
Step 3: Detailed Explanation:
First, we determine the size of the training dataset. - Total number of samples = 1000.
- Number of samples for testing = 100.
- Therefore, the number of samples available for training and model selection is: \[ \text{Training data size} = 1000 - 100 = 900 \] LOOCV is performed on this training dataset of 900 samples.
According to the definition of LOOCV, the number of validation splits is equal to the number of samples in the dataset it is applied to.
Since the training data has 900 samples, LOOCV will create 900 validation splits. In each split, 1 sample will be used for validation, and the remaining 899 samples will be used for training.
Step 4: Final Answer:
The number of validation splits generated by LOOCV on the training data is 900.