Reproduce a model result from a clean environment

A notebook that runs in its author's environment may rely on packages, files or state that were never recorded. Reproduce the result in an isolated environment to test whether the declared dependencies and artifacts are sufficient.

We performed that check for the original inactivity evaluation lab. A new Python3.12.0 virtual environment excluded system site packages, installed the recorded dependencies and reproduced the reference predictions. Maximum absolute prediction difference was approximately8.33e-17, below the declared1e-10 tolerance.

Freeze the reference before rebuilding anything

Keep the original input CSV, contract, source/code hashes and predictions. The reproduction script compares against them without overwriting them. Regenerating both the result and its expected output in the same step would provide much weaker evidence.

The script checks exact snapshot order and targets, selected C, split counts, input/code hashes, validation/test log losses and predicted probabilities. It also runs sixteen contract and split checks. This combines provenance with numerical comparison.

Recreate the environment from the project root

The following PowerShell commands show the structure used for this workspace. The virtual environment stays in the local tooling directory, outside the published teaching assets.

python -m venv .playwright-mcp/ds-evaluation-venv
.\.playwright-mcp\ds-evaluation-venv\Scripts\python.exe -m pip install -r 11-Blog-Programme/labs/ds-evaluation/requirements.txt
.\.playwright-mcp\ds-evaluation-venv\Scripts\python.exe -m pip check
.\.playwright-mcp\ds-evaluation-venv\Scripts\python.exe 11-Blog-Programme/labs/ds-evaluation/reproduce.py

For your own reproduction, choose a new environment path if that directory already contains a different experiment. Use the environment's Python executable explicitly; an activated terminal is convenient but should not be the only evidence of which interpreter ran.

The requirements file pins NumPy2.4.4, pandas3.0.2, SciPy1.18.0, scikit-learn1.9.0 and their recorded runtime dependencies. During setup, the dependency resolver initially selected a newer Narwhals release; we then pinned the reference version2.22.1 before performing the reported reproduction. The final dependency check found no broken requirements.

Read the recorded result with its scope

The clean-environment report records virtual_environment=true, system_site_packages=false, the installed versions and the maximum prediction difference. All sixteen contract checks passed.

This establishes reproduction on the tested Windows/Python environment and package builds. It does not establish identical results on every operating system, processor, numerical backend or future package release. The minimal requirements cover model evaluation; regenerating figures additionally requires the documented plotting dependencies.

Small floating-point differences can occur across numerical implementations. A tolerance should be chosen before comparison and tied to the result's meaning. Do not loosen it repeatedly until a materially changed result passes.

Investigate mismatches in a useful order

First check input and code hashes. Next inspect row order, feature order, split membership and package versions. Then examine randomness, solver settings and numerical behavior. A changed target label or preprocessing fit boundary deserves a different response from a harmless last-bit floating-point difference.

Keep failed reproduction evidence and explain the correction. Replacing the reference with the new result without understanding the mismatch would make the check unable to detect drift.

Exercise: reproduce the reference in a separate environment and submit interpreter details, dependency versions, the comparison output and any deviations. Then deliberately change the input CSV in a copy and confirm that the provenance check fails before accepting its score.

NeuraPath's Data Science course connects model results with repeatable workflows. A portfolio becomes more credible when another person can reproduce its evidence without inheriting an undocumented local setup.

Continue learning

This article is part of the Machine learning workflow and evaluation sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Document dataset provenance in an experiment manifest.
• Continue with Write a model card for a portfolio project.