Shadow deployment before switching model traffic

Shadow deployment sends a copy of production-like requests to a candidate while the champion continues to control responses. It can reveal schema, latency and prediction differences without changing user-facing decisions. It is not an online outcome experiment because the candidate does not act.

Compare two candidates without action

The local lab runs two regularization settings on 150 synthetic requests.

from deployment_cases import shadow_case

result = shadow_case()
assert result["traffic_actions_changed"] == 0
assert result["shadow_rows"] == 150
print(result["mean_absolute_score_difference"], result["decision_disagreement_rate"])

Mean absolute score difference is 0.0107 and threshold decisions disagree on 1.33% of requests. Candidate errors are zero in this in-process case. No network shadow service or real traffic is claimed.

Protect the primary path

Copy requests asynchronously where possible so candidate failure cannot slow or break the response. Apply the same authorization and privacy controls to the shadow path. Do not duplicate side effects such as messages, payments or database writes.

Log champion and candidate versions, input schema version, scores, decisions, latency and errors under a shared request ID. Sample safely if storing complete requests is not permitted.

Define promotion evidence

Set minimum traffic, duration and slices. Compare schema acceptance, missing values, latency, resource use, score distribution and disagreement. Join mature labels later for performance. Investigate disagreements rather than treating either model as automatically correct.

Shadow success supports compatibility and prospective evaluation. It does not prove an intervention improves outcomes. A canary with controlled actions and rollback comes later.

The Data Science course uses shadowing as one staged release tool with explicit limits.

Exercise

Design a shadow record and reconciliation query. Add a candidate timeout that cannot affect champion latency, then define the evidence required before a 5% canary.

Continue learning

This article is part of the Model deployment and MLOps sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Compare offline metrics with online decision outcomes.
• Continue with Canary releases with a measurable rollback condition.

Reference: Google Cloud model monitoring overview.