Pipelines and Leakage-Safe Preprocessing

Concept

Manual preprocessing is repetitive and easy to get wrong: split, impute, encode, scale, then model. A scikit-learn Pipeline chains these steps so training learns transformations once and inference applies the same learned transformations.

A Real Mixed-Column Pipeline

from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

numeric_features = ["age", "monthly_charges"]
categorical_features = ["contract_type", "region"]

numeric_transformer = Pipeline([
    ("imputer", SimpleImputer(strategy="median")),
    ("scaler", StandardScaler()),
])
categorical_transformer = Pipeline([
    ("imputer", SimpleImputer(strategy="most_frequent")),
    ("encoder", OneHotEncoder(handle_unknown="ignore")),
])
preprocessor = ColumnTransformer([
    ("numeric", numeric_transformer, numeric_features),
    ("categorical", categorical_transformer, categorical_features),
])
model = Pipeline([
    ("preprocessor", preprocessor),
    ("model", LogisticRegression(max_iter=1000)),
])

model.fit(X_train, y_train)
predictions = model.predict(X_test)

ColumnTransformer sends numeric and categorical fields down appropriate paths. During fit, each step learns only from X_train; during predict, raw new rows receive the identical learned preparation before the classifier runs.

Why Pipelines Prevent Leakage

Wrong: scaler.fit_transform(X_all) before cross_val_score. Every fold then contains scaling information from rows that should be held out. Right: pass the pipeline itself into cross-validation. Each fold fits its imputer, scaler, and encoder only on that fold's training portion.

Pipelines also prevent train/serve inconsistency: inference receives raw feature rows, not a manually reconstructed set of transformed columns. In GridSearchCV, pipeline parameters can be named model__C, keeping tuning connected to the same safe workflow.

Failure Signals

Common Mistakes

Preprocessing all rows before cross-validation.
Forgetting different treatment for categorical columns.
Refitting preparation on inference data.
Leaving target or post-outcome fields in X.

Practice Questions

What statistics does the numeric imputer learn during each CV fold?
Why use handle_unknown="ignore"?
Why is fit_transform(X_all) before CV unsafe?
Which pipeline stage runs first during prediction?
What does model__C refer to in tuning?

Module 5 Synthesis

Raw data -> split -> training-only preprocessing -> feature engineering -> pipeline -> cross-validation/tuning -> final model -> inference. Preprocessing is part of modeling, not a casual step before it.

Key Takeaway

Key Takeaways

Pipelines make mixed-data preparation repeatable, consistent, and leakage-safe across training, validation, and inference.

Next Lesson

Next, begin Module 6 and see how tree and ensemble models handle features differently.