Create a synthetic policy corpus for RAG practice

Real organizational documents can contain confidential or personal information and complex permissions. A synthetic corpus lets learners test ingestion, retrieval and failure handling without copying restricted content. It must be labelled synthetic and should contain realistic boundaries rather than invented performance claims.

Verify the corpus contract

The RAG ingestion lab creates four invented policy records.

from ingestion_cases import synthetic_corpus_case

result = synthetic_corpus_case()
assert result["documents"] == 4
assert result["unique_ids"] is True
assert result["invented"] is True
assert result["contains_personal_data"] is False
assert result["slices"] == ["public", "staff"]

The corpus includes two refund versions, travel rules and public help. Names, owners, dates and text are fictional. It cannot establish performance on real enterprise documents.

Design failure cases deliberately

Include current and superseded versions, contradictory clauses, missing evidence, a table, scanned-page fixture, duplicate copy, stale source, unsupported language, access boundary and indirect prompt injection. Attach an inventory row and expected retrieval evidence to every question.

Keep answers derivable from the corpus. Label unanswerable questions explicitly. Add source spans and reviewer notes so retrieval and generation can be evaluated separately.

Make generation reproducible

Version the generator or source files, use stable IDs and save digests. If an LLM helps draft synthetic documents, review them for accidental names, copied phrases and internal contradictions; record that process. Prefer clearly fictional domains and addresses.

Do not call synthetic documents anonymized real documents unless that transformation has been verified. Synthetic data reduces some privacy risk but does not prove a production system handles real layout, language or distribution.

The Generative & Agentic AI course uses synthetic practice before any approved domain corpus and requires the limitations in the evaluation report.

Exercise

Author ten policy documents with two version families and eight failure slices. Create 50 question/evidence labels, validate unique IDs and publish a manifest that states origin, licence and synthetic limitations.

Continue learning

This article is part of the RAG ingestion and document preparation sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Validate document encoding and language before indexing.
• Continue with Protect secrets and personal data during ingestion.

Reference: NIST privacy engineering resources.