P83Machine LearningAdvancedDesign18 min
Design a Time-Aware Validation Strategy
Design chronological validation for a system that predicts future observations.
#design#time-aware validation#deployment
Scenario
A subscription-risk model trains on monthly account snapshots from 2024 through 2026 and will score next month's accounts.
Problem statement
Propose a validation structure that reflects future-facing deployment and explain when a random split would be unrealistic.
Your Task
- Define training and validation periods.
- Keep validation after training chronologically.
- Explain information availability and distribution shift.
Related Concepts
Temporal validationDistribution shiftLeakage
Ready to count this problem?
Mark it complete when you have reasoned through the solution in your own words.