XGBoost validation: early stopping without test-set leakage

The dataset that chooses when boosting stops is model-selection data. If it is your final test set, the stopping decision has already used that test information. Use a validation set for stopping and assess the frozen result separately.

Our original synthetic duration exercise trains on 160 rows, monitors 80 validation rows and assesses 80 test rows afterward. Only validation enters eval_set. The result demonstrates the mechanics of early stopping, not production performance or a universal best configuration.

Declare the stopping configuration

The model uses depth-two trees, learning rate 0.1, histogram tree construction, a maximum of 600 rounds and patience of 20 rounds. The evaluation metric is RMSE, with a fixed seed and one worker thread.

The XGBoost estimator guide documents early stopping through the evaluation set and the estimator's use of the best iteration for predictions. Our local run uses XGBoost 3.4.1; the checked stable documentation identifies 3.4.2, which was unavailable from the package index used for this installation.

Dataset variable names are not sufficient evidence of their role. Even if an example calls its monitored data “test,” using it for stopping makes it validation in the experiment design.

Read the actual stopping result

Best iteration is 80 under the zero-based convention, corresponding to 81 selected trees. The booster retains 101 trained rounds after the patience period. Best validation RMSE is approximately 1.627308, and test RMSE using the selected range is approximately 2.040212.

The distinction between retained rounds and selected prediction rounds matters when switching from the scikit-learn wrapper to the native booster interface.

import numpy as np
from boosting_cases import xgboost_case

r = xgboost_case()
assert r['version']=='3.4.1'
assert r['evaluation_sets']==['validation_0']
assert r['best_iteration_zero_based']==80
assert r['selected_tree_count']==81 and r['trained_rounds']==101
assert r['best_index_matches_recorded_minimum']
assert r['sklearn_native_best_range_agree']
assert r['json_roundtrip_best_range_agrees']
assert np.isclose(r['test_rmse_using_best_iteration'],2.0402115479422065)
print({key:value for key,value in r.items() if key!='validation_history'})

Use the extension interpreter described in the lab README. The included requirements pin the executed package versions, and the full validation history is recorded in the verification report.

Verify the prediction range after serialization

The native comparison explicitly requests iteration_range=(0,best_iteration+1). The upper endpoint is exclusive. Using (0,80) would omit the selected last tree; using every retained round would describe a different prediction function.

The lab also serializes the booster to JSON bytes and reloads it in memory, then compares predictions using the same selected range. This checks the specific roundtrip, not every possible deployment wrapper or future package version.

Keep further tuning inside the development process

Patience, depth, learning rate and other choices still influence the fitted model. Trying many configurations against the same validation set is a search procedure and should be recorded. Early stopping does not remove the need for a suitable final assessment.

The dataset is shared across instructional algorithm cases. Its scores are useful worked examples, but repeated exploration across the series is not an untouched production benchmark for selecting a model family.

Exercise: declare a different patience value before running the comparison. Record best iteration, trained rounds and validation history, then verify the native prediction range. Explain which data informed the choice and how you would assess a final procedure on new evidence.

NeuraPath's Data Science course connects boosting APIs with evaluation discipline. A complete early-stopping result preserves the data role, indexing convention and prediction range that produced its score.

Continue learning

This article is part of the Supervised learning methods sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Gradient boosting: why weak learners can overfit together.
• Continue with LightGBM categorical features: verify the encoding assumptions.