Feature Scaling and Transformations

Concept

Income near 80,000 and age near 35 have very different ranges. Scale matters for distance-based, regularized, and gradient-based models because large-number features can dominate calculations. It does not affect every model equally: tree-based models usually do not depend on scale in the same way.

Standardization and Min-Max Scaling

StandardScaler learns the training mean and standard deviation, then centers and scales numeric features. MinMaxScaler maps a training range to a chosen range, commonly 0 to 1. Neither is universally best; choose based on model behavior and data context.

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

Fit once on training data. Fitting separately on test data changes feature meaning and leaks held-out distribution information. Scaling is especially relevant to Ridge/Lasso because regularization reacts to coefficient magnitude.

Transformations

Income, transaction amount, and house price can be strongly right-skewed. A log transformation can compress extreme ranges and make a relationship easier for a model to represent. It changes interpretation and does not automatically fix outliers, missingness, or a weak target.

Failure Signals

Common Mistakes

Claiming every model needs scaling.
Fitting scalers on all rows.
Applying a log transformation without considering zero or negative values.
Treating transformed values as original units.

Practice Questions

Why might age and income need scaling for logistic regression?
Why might a tree not need it?
What data belongs in scaler.fit()?
What does a log transform change?

Key Takeaway

Key Takeaways

Scaling is model-dependent and must be learned from training data. Transformations change representation, not the underlying data-quality problem.

Next Lesson

Next, create useful signals while avoiding leaky or unnecessary features.