Generative AI & Agentic AIRAG ingestion and document preparation

Reindex a corpus without disrupting active readers

PK
Pankit Kumar
Sr. Data Scientist at Parexel (a Goldman Sachs–backed company) · 20 September 2026 · 2 min read
Technically reviewed by Ishaan Sharma
In this article (5 sections)

Changing chunking or embeddings rewrites the retrieval space. Updating an active collection in place can mix versions and make rollback uncertain. Build a complete candidate index, validate it and switch through a stable alias or application pointer.

Simulate blue/green indexes

The RAG ingestion lab uses two local metadata records.

python
from ingestion_cases import reindex_case

result = reindex_case()
assert result["active_before"] == "blue"
assert result["active_after"] == "green"
assert result["old_retained"] is True
assert result["reader_downtime_simulated"] is False
print(result["pattern"])

Both indexes contain four synthetic documents and pass the fixture validation, so the pointer moves to green while blue remains available. This is a state simulation, not a live availability test.

Build the candidate independently

Freeze the source manifest and configuration. Write to a new collection with versioned chunk/embedding identities. Verify expected documents, chunks, metadata, vector dimensions and deletion tombstones. Run labelled retrieval and authorization tests before switching.

Shadow current queries against both versions where policy allows. Compare evidence IDs, latency and errors without using shadow results to expose unauthorized content. Define critical regressions that block promotion.

Switch and observe

Use an atomic alias operation where supported or a transactional application pointer. Record approver, time and versions. Monitor retrieval failures and answer-support gates after the switch. Keep the old index read-only for a bounded rollback window.

Rollback restores the previous complete pointer; it must also respect deletions and authorization changes made during the window. Rehearse it. Remove the old index only after acceptance and retention conditions are met.

The Generative & Agentic AI course links reindexing to source manifests, versioned embeddings, eval gates and operational runbooks.

Exercise

Build two local collections with different chunkers, validate counts and labelled retrieval, switch an alias, inject a regression and roll back. Document what real-service availability evidence is still missing.

Continue learning

This article is part of the RAG ingestion and document preparation sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

Reference: Qdrant collection aliases.

PK
Pankit Kumar
Lead Instructor, NeuraPath Academy

Pankit Kumar has 10 years in Data Science & AI, building and shipping production systems in regulated pharma and clinical environments. He is a freelance trainer at Boston Institute of Analytics, AnalytixLabs and Scaler, and has taught this material to thousands of working professionals.

This article is part of our Generative & Agentic AI programme — 3 months. Add practical GenAI, retrieval and agent-building skills to your existing toolkit.

Explore Generative & Agentic AI
Counselling is free · no obligation

Not sure which programme fits?

Tell us your background and we will map it to the right entry point — including saying so when a cheaper programme is the better fit. A counsellor replies within one working day.