Why Accuracy Is Not Enough

The problem with accuracy

Suppose a fraud dataset contains:

• 990 normal transactions
• 10 fraudulent transactions

A model predicts every transaction as normal.

Its accuracy is:

990 / 1000 = 99%

The number looks excellent, but the model detected zero fraud cases.

This is why evaluation must reflect the decision you are trying to make.

Precision

Precision asks:

Of everything predicted as positive, how much was actually positive?

High precision is important when false positives are costly.

Example: an email system should avoid placing legitimate business emails into spam.

Recall

Recall asks:

Of all actual positive cases, how many did the model detect?

High recall matters when missing a positive case is costly.

Example: a fraud screening system may tolerate additional investigations if it helps capture more actual fraud.

F1 score

F1 combines precision and recall into a single harmonic-mean score. It is useful when you need a balance between the two, especially with imbalanced classes.

ROC-AUC

ROC-AUC evaluates how well a model ranks positive examples above negative examples across thresholds. It can be useful for comparing ranking ability, but it should not automatically replace business-relevant threshold metrics.

For highly imbalanced problems, also inspect the precision-recall curve and metrics at the threshold you actually plan to use.

Python example

from sklearn.metrics import classification_report

print(classification_report(y_test, y_pred))

Do not stop at one number. Inspect the metrics for the class that actually matters.

Quick decision guide

Situation	Metric to inspect closely
Balanced classes, equal error cost	Accuracy can be useful
False positives are costly	Precision
False negatives are costly	Recall
Need precision/recall balance	F1
Comparing ranking across thresholds	ROC-AUC

Common mistake

Choosing a metric before understanding the business cost of different errors.

The same model can be good for one operating objective and unacceptable for another.

Interview perspective

If asked “Which metric would you use for fraud detection?”, do not answer with a metric name only. Ask or explain which error is more costly, how imbalanced the data is, and how predictions will be acted upon.

Practice

A medical screening model has 96% accuracy but only 58% recall for the disease class. What does this tell you? What would you investigate before deployment?