Preparing Missing and Categorical Data for Models

Concept

For ML, preparation is part of the model workflow. Many estimators cannot consume missing values or arbitrary category text directly, and every learned preparation choice must come from training data only.

Missing Values Without Leakage

Mean, median, and most-frequent imputation replace missing values with a learned reference. The critical order is split first, then fit the imputer on training data, then transform validation or test rows. Calculating a median from every row before splitting lets held-out distributions influence training preparation.

from sklearn.impute import SimpleImputer

imputer = SimpleImputer(strategy="median")
X_train_ready = imputer.fit_transform(X_train)
X_test_ready = imputer.transform(X_test)

Median is often more robust than mean for skewed income; most-frequent can suit a categorical field. Dropping rows can be reasonable when missingness is rare, but it may discard useful examples or systematically remove one group. Missingness itself can sometimes carry signal, such as an absent income field, but add indicators only with a plausible prediction-time reason.

Categorical Inputs

Models need numeric representations. Ordinal encoding fits categories with meaningful order, such as low/medium/high. One-hot encoding fits unordered categories such as city or payment method. Encoding red=1, blue=2, green=3 falsely suggests order and distance.

from sklearn.preprocessing import OneHotEncoder

encoder = OneHotEncoder(handle_unknown="ignore")
X_train_city = encoder.fit_transform(X_train[["city"]])
X_test_city = encoder.transform(X_test[["city"]])

handle_unknown="ignore" prevents a new inference-time city from crashing the transformation. It does not make unknown values meaningful; teams should still monitor data changes.

Practical Decision

For a churn table, impute missing monthly_charges from training data, one-hot encode contract_type, and keep churn only as y. A cancellation date is leaky because it is unavailable at prediction time.

Failure Signals

Common Mistakes

Imputing before splitting.
Giving unordered categories arbitrary numeric ranks.
Dropping missing rows without checking who is removed.
Fitting an encoder separately on test data.

Interview Perspective

Question: Why fit an imputer only on training data? Answer: held-out values would otherwise influence learned preparation. What the interviewer is testing: leakage awareness.

Practice Questions

Choose mean or median for heavily skewed income.
Should payment method use ordinal or one-hot encoding?
Why is an unknown category an inference concern?
Identify the leaky field in a pre-churn prediction table.

Key Takeaway

Key Takeaways

Preparation learns from training data, then applies unchanged to new rows. Missing and categorical values require decisions based on meaning and prediction timing.

Next Lesson

Next, learn when numeric scale and transformations affect a model.