P35Model EvaluationAdvancedInterpret18 min
Cost-Sensitive Model Selection
Choose between models with conflicting metrics using an explicit error-cost model.
#cost-sensitive#thresholds#business-metrics#classification
Scenario
For fraud, a false negative costs about 800 while a false positive costs 25 in manual review and customer friction. Two models have different precision and recall profiles.
Problem statement
Design a comparison that selects an operating threshold by expected cost rather than accuracy.
Your Task
- Define the cost for every confusion-matrix outcome that matters.
- Evaluate multiple thresholds on validation data.
- Check the final choice on a separate test period.
Related Concepts
Expected valueThreshold tuningDecision analysis
Ready to count this problem?
Mark it complete when you have reasoned through the solution in your own words.