Handle scanned documents with OCR quality checks

OCR can turn a visible rule into plausible but wrong text: 12000 becomes 1200, ID becomes 1D, or a negation disappears. A RAG pipeline needs an OCR quality gate before it treats extracted text as evidence.

Measure against ground truth

The RAG ingestion lab compares three authored OCR strings with known text.

from ingestion_cases import ocr_quality_case

result = ocr_quality_case()
assert result["threshold"] == 0.08
assert result["failed_pages"] == [3]
assert result["ocr_engine_executed"] is False
print(result["pages"])

Page 3 exceeds an 8% character-error threshold and enters review. No OCR engine ran, so the result demonstrates gate arithmetic rather than engine quality.

Sample by document difficulty

Create human-verified reference text for representative pages: clean print, skew, handwriting, stamps, tables, low contrast and multiple scripts. Calculate character or word error rates with clear normalization. For field extraction, measure exact field accuracy because a low overall error can still corrupt a critical amount or date.

Record engine/version, language configuration, preprocessing and page image identity. Compare versions on the same page set. Do not report one corpus-wide number without document-type denominators.

Route uncertain pages deliberately

Store OCR confidence only as a signal; calibrate it against observed errors. Quarantine pages that fail thresholds, flag uncertain spans and prevent unsupported citations. A human correction needs reviewer identity and a link to the original image. Preserve both raw and corrected text according to access policy.

Scan ingestion must also handle empty pages, orientation, duplicate scans, partial pages and embedded text layers that disagree with the visible image. Treat a missing OCR result as a processing failure, not an empty policy.

The Generative & Agentic AI course separates OCR evidence from downstream answer quality and includes exception review in multimodal/RAG work.

Exercise

Build a 30-page labelled sample from permitted scans. Measure overall and critical-field accuracy, set a quarantine rule, then test whether failed pages can ever reach retrieval or citations.

Continue learning

This article is part of the RAG ingestion and document preparation sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Extract PDF text while preserving page references.
• Continue with Chunk documents around meaningful boundaries.

Reference: Tesseract user manual.