Document a data dependency that can invalidate a model

A model artifact can remain byte-for-byte unchanged while its predictions become invalid. An upstream team may change a unit, event definition, timestamp or late-data policy. Those changes alter the population or feature meaning before conventional model-performance monitoring has mature labels.

Document critical data dependencies as operating contracts tied to a response.

Use a dependency record that can drive action

The local data-engineering lab defines a dependency for settled payment events.

from data_engineering_cases import dependency_case

result = dependency_case()
assert result["complete"] is True
dependency = result["dependency"]
assert dependency["unit"] == "integer cents"
assert dependency["event_time"] == "settled_at"
print(dependency["response"])

The record names the dependency and owner, source field, unit, event-time meaning, availability target, quality gates, breaking examples, model impact and response. Its quality gates require a unique event ID, nonnegative amount and settled status. A unit change, status-definition change or late-event-policy change can alter spend_30d even when the source remains syntactically valid.

Connect source failure to model impact

For every feature, trace source → validated staging data → transformation → feature → model version. Add the prediction populations and time windows affected by each edge. This makes questions answerable during an incident: which deployed models consumed the field, which training sets need rebuilding, and from what date?

Define failure signals at the earliest useful layer. Freshness and missing partitions belong near ingestion. Unit, enumeration and join-cardinality checks belong in transformed data. Feature-distribution alerts provide another signal, but they should not be the first place a known contract break appears.

Avoid vague responses such as “monitor closely.” The lab’s response blocks feature publication and model refresh, then notifies data and model owners. A deployed service may also need to fall back to an approved model, abstain, or disable an affected feature according to a pretested runbook. Name who can make that decision and what evidence restores service.

Review dependency changes before retraining

Attach effective dates and contract versions to producer changes. Evaluate old and new data over an overlap window. Compare row membership, missingness, units, distributions and representative records; then rerun the model evaluation that is sensitive to the change. Store the accepted dependency version in the training and deployment manifests.

The Data Science course treats this record as part of model validity, alongside split design and held-out evaluation. It is also a handover artifact: an operations owner can see why an upstream change matters without reverse-engineering a notebook.

Exercise

Document one dependency for a churn, fraud or demand model. Inject a one-day delay, a cents-to-rupees change and a status-enumeration change. Show which checks fail, identify affected features and write the exact block, fallback and recovery criteria.

Continue learning

This article is part of the Data engineering for data science sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the preceding task in Build a reproducible data pipeline on a local machine.
• Return to the cluster foundation in ETL versus ELT through a model-training pipeline.

References: Google’s data validation paper and OpenLineage facets documentation.