Build a RAG document inventory with source ownership

A retrieval system cannot manage what it has never inventoried. Before chunking or embedding, record where each source came from, who can approve it, which version applies and who may retrieve it.

Create one row per source version

The RAG ingestion lab uses four original synthetic policies.

from ingestion_cases import inventory_case

result = inventory_case()
assert result["count"] == 4
assert result["missing_owner"] == []
assert result["owners"] == ["finance", "people", "support"]
assert all(len(row["sha256"]) == 64 for row in result["documents"])

Each row records document ID, title, owner, version, effective date, access class and SHA-256 digest. Two refund policies remain separate versions. The files and owners are invented and do not represent NeuraPath operations.

Record lifecycle fields

A production inventory should also include source URI or system ID, ingestion permission, format, language, jurisdiction, retention rule, supersedes relationship, review date, deletion status and downstream indexes. Treat the digest as content identity, not proof that the content is correct or authorized.

Assign two kinds of owner where needed: a business owner responsible for meaning and a technical owner responsible for ingestion. Define who can mark a version effective, resolve a contradiction and approve deletion.

Gate indexing from the inventory

Reject or quarantine sources without ownership, access policy, effective version or parseable content. Filter authorization before retrieval and recheck it at answer time. A public help document and staff policy should not enter the same unrestricted result merely because their text is similar.

Use the inventory for change detection. A new digest under an unchanged version deserves investigation; a new approved version triggers ingestion and evaluation while preserving the prior record for reproducibility. Link every chunk and embedding back to the inventory row.

The Generative & Agentic AI course makes the source inventory a required RAG artifact before vector-database work.

Exercise

Inventory 20 permitted documents. Fail the build when owner, access or effective date is absent. Change one file without changing its version, confirm the digest alert, and trace every generated chunk back to its source row.

Continue learning

This article is part of the RAG ingestion and document preparation sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Continue with Extract PDF text while preserving page references.
• Then apply it in Handle scanned documents with OCR quality checks.

Reference: NIST Generative AI Profile.