Handle contradictory document versions explicitly

Retrieving two policy versions can produce an answer that blends incompatible rules. Version metadata and as-of logic must decide which source applies—or force the system to surface uncertainty.

Detect a changed rule

The RAG ingestion lab compares two synthetic refund policies.

from ingestion_cases import contradiction_case

result = contradiction_case()
assert result["version_ids"] == ["refund-v1", "refund-v2"]
assert result["contradiction_detected"] is True
assert result["selected_as_of_2026_09_20"] == "refund-v2"
assert result["silent_merge"] is False

Version 1 requires a receipt; version 2 allows a receipt or verified order ID. The later effective version applies on the fixture date. The older document remains for historical queries.

Model version relationships

Store stable policy family, version, effective interval, approval state and supersedes link. Detect text changes at clause level and route material differences to the owner. Similarity alone cannot decide whether a change is meaningful.

Current queries should filter to the approved version effective at the decision date. Historical questions should specify an as-of date. If two approved sources overlap or disagree without precedence, abstain or present the conflict with both citations according to the application contract.

Evaluate contradiction behaviour

Create question pairs whose answers change across versions. Test current, historical, missing-date and overlapping-effective-period cases. Measure whether retrieval supplies the right source and generation preserves the exact distinction.

Never merge chunks from multiple versions into one source identity. Keep deletion and access changes consistent across the family. A new version also triggers regression evaluation because altered evidence can change valid answers without any prompt/model change.

The Generative & Agentic AI course connects version resolution to freshness, citations and abstention policies.

Exercise

Author three policy versions with one changed threshold and one unchanged section. Implement as-of filtering, inject an overlap conflict and verify the assistant abstains rather than composing a hybrid rule.

Continue learning

This article is part of the RAG ingestion and document preparation sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Measure ingestion completeness with a source manifest.
• Continue with Write a data contract for a knowledge assistant.

Reference: NIST Generative AI Profile.