Creating and Selecting Useful Features

Concept

Feature engineering uses existing prediction-time information to create representations that make a useful pattern easier for a model to learn. Feature selection decides which signals are worth retaining. More columns do not automatically create a better model.

Create Signals From Raw Data

A transaction date can become day of week, month, or weekend indicator. Purchase history can become days since last purchase, purchase frequency, or average order value. These features may expose behavior that one raw timestamp or order row hides.

Domain knowledge matters: a retention team may know that recent support activity relative to account age is more informative than either field alone. Interactions can also help, such as rooms relative to house size. Every created feature must exist when the future prediction is made.

Leakage Check

If price is the target, price_per_area leaks the target and must not be used as X. A post-cancellation refund or a support note written after churn is similarly invalid. Ask: would this value genuinely exist for a new row at prediction time?

Select Deliberately

Drop pure identifiers, duplicates, leaky columns, extremely noisy fields, and inputs too costly to obtain. Correlated features are not automatically wrong; they may improve prediction while making linear coefficients unstable. Choose with domain reasoning, simple screening, and model evidence on development data, never by repeatedly inspecting final test results.

Practical Triage

Candidate	Decision	Reason
`signup_date` -> tenure	Transform	Available and more directly useful.
`customer_id`	Usually drop	Identifies a row, not a stable behavior.
`cancellation_reason`	Drop	Known after churn.
`support_calls / tenure`	Test carefully	May express a useful interaction.

Failure Signals

Common Mistakes

Creating target-derived features.
Keeping every available column.
Removing every correlated feature automatically.
Selecting features using final test performance.

Interview Perspective

Question: What is feature engineering? Answer: creating valid representations from available information that help a model learn useful patterns. What the interviewer is testing: domain reasoning and leakage awareness.

Practice Questions

Turn an order date into two useful features.
Why is price per area invalid when price is y?
Keep, transform, or drop a customer ID?
Why can more features increase overfitting risk?
What makes a feature valid at inference time?

Key Takeaway

Key Takeaways

Useful features combine domain understanding with strict prediction-time availability. Select signals for value and stability, not column count.

Next Lesson

Next, make preprocessing repeatable and leakage-safe with pipelines.