Build a scikit-learn pipeline that prevents preprocessing leakage

A pipeline prevents a common form of leakage when you fit the entire pipeline only on the training portion of each split. It keeps learned preprocessing and the estimator together, so validation data receive the fitted transformations without contributing to their parameters.

The guarantee depends on how you call it. Fitting a pipeline on the full dataset before splitting still exposes the transformations and model to held-out data. A pipeline also cannot repair a feature that encodes an outcome unavailable at prediction time.

Assemble the transformations around the data contract

Our original synthetic inactivity model uses three numeric features and one category. The numeric branch imputes missing values with training medians, adds missingness indicators and scales the resulting columns. The categorical branch one-hot encodes the plan. Logistic regression consumes the combined representation.

The feature list excludes future active days, target labels, identity fields and split metadata. The lab implementation uses a ColumnTransformer with remainder='drop', inside a Pipeline with the estimator.

Scikit-learn's common-pitfalls guide documents training-only transformation fitting. The original dataset and recorded checks below let you inspect that boundary directly.

Inspect what was learned

January training data contain nine missing ticket values among 160 rows. The numeric training medians are 7 for days since activity, 1 for tickets and 126.5 for tenure. Across all 320 snapshots, tenure's median is 148.5. Using that full-data statistic would incorporate later periods.

import numpy as np
from evaluation_core import load,build_model,FEATURES,NUMERIC,TARGET

d = load()
train = d[d['split']=='train']
test = d[d['split']=='test']
model = build_model(C=.1).fit(train[FEATURES],train[TARGET])
numeric = model.named_steps['prepare'].named_transformers_['numeric']
medians = numeric.named_steps['impute'].statistics_.copy()
assert np.allclose(medians,[7,1,126.5])
assert np.allclose(medians,train[NUMERIC].median().to_numpy())
assert d['tenure_days'].median()==148.5
before = numeric.named_steps['scale'].mean_.copy()
p = model.predict_proba(test[FEATURES])[:,1]
assert np.isfinite(p).all() and len(p)==80
assert np.array_equal(before,numeric.named_steps['scale'].mean_)
assert np.array_equal(medians,numeric.named_steps['impute'].statistics_)
print({'training_medians':medians.tolist(),'test_predictions':len(p),
       'prediction_refitted_preprocessing':False})

Run from the evaluation lab. The check verifies that prediction leaves the fitted medians and scaling means unchanged. It does not claim that the chosen preprocessing is optimal.

Preserve the boundary during cross-validation

Pass the full unfitted pipeline to the cross-validation or hyperparameter-search procedure. Each training fold should learn its own imputation, scaling and encoding. Precomputing transformed features on all development rows before cross-validation would let each held-out fold influence the shared representation.

If you need grouped or time-aware folds, supply those deliberately. The pipeline manages estimator operations; the splitter determines which observations are allowed to influence them. Both choices matter.

Handle missing and unseen values deliberately

An imputed ticket count of one is an estimation choice, not evidence that the account actually had one ticket. The missingness indicator retains information that the count was absent. Investigate why values are missing and whether the pattern changes later.

The category encoder ignores unknown categories in this reference, allowing transformation to complete. That is a runtime policy, not evidence of reliable predictions for a new plan. Report unseen-category rates and evaluate the affected slice before making a reliability claim.

Keep the remaining leakage checks explicit

Audit source availability, target construction, duplicate prediction instances and entity/time boundaries outside the pipeline. A perfectly fitted scaler cannot make a future-derived field legitimate.

Exercise: deliberately fit a separate imputer on all rows and record the changed tenure median. Then restore the training-only pipeline and add a validation case with an unseen plan. Explain both the transformation behavior and the evidence still needed to trust that category's predictions.

NeuraPath's Data Science course connects reproducible pipelines with honest evaluation. Inspecting fitted statistics turns “we avoided preprocessing leakage” into a claim that a reviewer can check.

Continue learning

This article is part of the Machine learning workflow and evaluation sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Time-aware validation for a changing business process.
• Continue with Compare models with uncertainty instead of one lucky score.