Model retraining triggers based on evidence

Retraining is an experiment, not a repair command. New data can be mislabeled, shifted or incompatible. A good policy distinguishes investigation, candidate training and production promotion.

A small decision matrix

The local deployment lab encodes four evidence states.

from deployment_cases import retraining_case

result = retraining_case()
assert result["automatic_production_promotion"] is False
for signal in result["signals"]:
    print(signal)

Input drift without performance labels triggers investigation only. Confirmed performance drop with fresh labels starts retraining evaluation. Fresh labels with stable performance do not force retraining. No candidate is promoted automatically.

Define trigger eligibility

Set minimum mature labels, evaluation window, degradation margin and persistence. Include critical slices and calibration. Check data quality and target definition before fitting. A broken upstream unit can make “fresh data” harmful.

Calendar retraining can be reasonable when data and operations change predictably, but it still needs candidate evaluation. Event-based retraining responds to evidence; it should include cooldowns and ownership to avoid repeated jobs from noisy alerts.

Keep release gates intact

A retrained model must beat the approved baseline on fixed validation, pass schema and reload tests, satisfy latency and fairness/safety constraints where relevant, and enter staged release with rollback. Preserve the current production model until those gates pass.

Track whether retraining resolved the triggering failure. Otherwise the organization has automated compute rather than improvement.

The Data Science course treats retraining as governed model selection with evidence.

Exercise

Write thresholds for one delayed-label project. Simulate noisy drift, real degradation and label-pipeline failure. Verify which states investigate, retrain or stop the pipeline.

Continue learning

This article is part of the Model deployment and MLOps sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Design delayed-label monitoring for a deployed model.
• Continue with Create a model rollback runbook.

Reference: Google Cloud MLOps continuous training guidance.