Question

Consider that 20 stories of author X and 10 stories of author Y were kept together without mentioning the names of the authors. A classifier was then asked to predict the author (X or Y) of each of the stories. Later out of X's stories 6 were classified as that of Y. On the other hand, out of Y's stories 2 were classified as that of X. Considering X and Y as two classes, then which of the following is/are true?

• A. Recall of class X is higher than the recall of class Y.
• B. Precision of class X is higher than the precision of class Y.
• C. Accuracy of the classifier is 11/15.
• D. Accuracy of the classifier is 14/15.

Hint:

Drawing a confusion matrix is the most reliable way to solve classification metric problems. Remember the definitions: - f{Accuracy:} (TP+TN)/Total (Overall correctness).
- f{Precision:} TP/(TP+FP) (Of all positive predictions, how many were right?).
- f{Recall (Sensitivity):} TP/(TP+FN) (Of all actual positives, how many did we find?).

Answer 1

The Correct Option is B, C

Step 1: Understanding the Question:
We are given the results of a binary classifier and need to calculate and compare performance metrics like recall, precision, and accuracy. We can organize the given information into a confusion matrix. Let's consider 'X' as the positive class.
Step 2: Constructing the Confusion Matrix:
- Total actual X stories = 20 (Total Positives, P)
- Total actual Y stories = 10 (Total Negatives, N)
- Total stories = 30
From the results: - "out of X's stories 6 were classified as Y": These are actual X stories that were incorrectly predicted as Y.
This is the number of False Negatives (FN). FN = 6.
- The number of actual X stories correctly predicted as X is the number of True Positives (TP).
TP = Total Actual X - FN = 20 - 6 = 14.
- "out of Y's stories 2 were classified as that of X": These are actual Y stories that were incorrectly predicted as X. This is the number of False Positives (FP). FP = 2.
- The number of actual Y stories correctly predicted as Y is the number of True Negatives (TN). TN = Total Actual Y - FP = 10 - 2 = 8.
The confusion matrix (Rows: Actual, Columns: Predicted) is:

Step 3: Calculating and Evaluating the Metrics:
- (A) Recall(X) vs Recall(Y):
- Recall(X) = $\frac{\text{TP}}{\text{Actual X}} = \frac{14}{20} = 0.70$
- Recall(Y) = $\frac{\text{TN}}{\text{Actual Y}} = \frac{8}{10} = 0.80$
- Recall(X) is not higher than Recall(Y). So, (A) is False.
- (B) Precision(X) vs Precision(Y):
- Precision(X) = $\frac{\text{TP}}{\text{Predicted X}} = \frac{14}{14+2} = \frac{14}{16} = 0.875$
- Precision(Y) = $\frac{\text{TN}}{\text{Predicted Y}} = \frac{8}{6+8} = \frac{8}{14} \approx 0.571$
- Precision(X) is higher than Precision(Y). So, (B) is True.
- (C) Accuracy:
- Accuracy = $\frac{\text{TP + TN}}{\text{Total}} = \frac{14 + 8}{30} = \frac{22}{30} = \frac{11}{15}$
- So, (C) is True.
- (D) Accuracy:
- This states accuracy is 14/15, which contradicts (C). So, (D) is False.
Step 4: Final Answer:
The true statements are (B) and (C).