Preserve tables during document ingestion

A table cell rarely stands alone. The value manager may mean an approval level only when paired with a distance band and travel mode. Flattening cells without headers can create a convincing but unsupported answer.

Keep row and column relationships

The RAG ingestion lab defines a two-row synthetic travel table.

from ingestion_cases import table_case

result = table_case()
assert result["columns"] == ["distance_km", "mode", "approval"]
assert result["row_count"] == 2
assert result["all_cells_preserved"] is True
print(result["flattened"])

The flattened representation repeats each distance band with its mode and approval. The check verifies content preservation on a simple fixture, not table extraction from arbitrary PDFs.

Store a structured form and a retrieval form

Keep the table as rows with headers, source page, table title, units, footnotes and merged-cell interpretation. Generate a text form for lexical or vector retrieval, but link it to the structured source. Stable row IDs allow field-level citations.

Large tables may need row groups. Repeat headers and relevant section context in each chunk. Do not separate footnotes or unit declarations from affected rows. If a table spans pages, verify that repeated headers are not ingested as data.

Evaluate exact fields

Create questions that target values, row conditions, ranges, units and footnotes. Measure whether retrieval returns the right table/row and whether extraction returns each field exactly. Compare against the rendered source for a labelled sample.

Route ambiguous merged cells or low-quality OCR to review. A model should abstain when the row relationship cannot be reconstructed. Preserve original images when users need visual confirmation and access policy permits it.

The Generative & Agentic AI course includes table preservation in ingestion and field-level metrics in multimodal evaluation.

Exercise

Ingest a permitted table with merged headers, units and a footnote. Produce structured rows and retrieval text, then test ten exact-value questions and prove every answer opens the correct page and row.

Continue learning

This article is part of the RAG ingestion and document preparation sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Compare chunk overlap with a retrieval benchmark.
• Continue with Attach metadata that supports retrieval filters.

Reference: pypdf extraction documentation.