P09Model EvaluationIntermediateDiagnose14 min
99% Accuracy, Zero Fraud Detected
Diagnose a fraud model that looks accurate overall but fails at the outcome the business cares about.
#class imbalance#recall#precision
Scenario
Only 1% of transactions are fraudulent. A model predicts every transaction as legitimate and reports 99% accuracy, but it catches no fraud.
Problem statement
Explain why accuracy is misleading, choose metrics to inspect next, and describe one practical modeling or evaluation change.
Your Task
- Identify the class imbalance problem.
- Explain what fraud recall is in this situation.
- Discuss the tradeoff between catching fraud and flagging legitimate customers.
Confusion-matrix summary
10,000 transactions
100 fraud, 9,900 legitimate
Model predicts all 10,000 as legitimate
Accuracy = 99%, fraud recall = 0%
Related Concepts
Precision and recallThreshold tuningCost-sensitive evaluation
Ready to count this problem?
Mark it complete when you have reasoned through the solution in your own words.