Extract structured fields from text and validate each field

Text extraction should produce a schema, not an unverified paragraph. Define fields, types, normalization, null behavior and provenance. A simple deterministic parser is often the right baseline for regular identifiers and dates.

Four invented records

The local NLP lab extracts order_id, date, amount and email with regular expressions from four invented messages.

from nlp_cases import extraction_case

result = extraction_case()
assert result["fields"] == ["order_id", "date", "amount", "email"]
assert result["schema_valid_dates"] == 4
for row in result["rows"]:
    print(row["prediction"])

Amounts such as INR 1,299 normalize to the string 1299. Missing order IDs, amounts and emails remain None; they are not invented. All four date strings match YYYY-MM-DD.

Format validation is not semantic validation. A string such as 2026-99-99 matches the basic pattern but is not a date. Production code should parse calendar values, validate currency and ranges, and preserve the source span.

Define each field contract

For every field, specify required or nullable status, multiplicity, type, normalization and allowed values. Decide how multiple order IDs are represented. Return structured errors for invalid input instead of silently choosing the first match.

Retain the original text and character offsets for audit. If a generative model performs extraction, constrain output with a schema, validate it and distinguish missing from model refusal or parse failure. Never treat syntactically valid JSON as factually correct.

Test boundaries

Include absent, repeated, malformed and conflicting fields; international number and date formats; punctuation; Unicode; and adversarial instructions if a language model reads untrusted text. Score each field separately because one easy field can hide another’s failure.

The Data Science course connects text extraction to data contracts and field-level evaluation.

Exercise

Add semantic date parsing, currency codes and multiple-order handling. Create 100 labelled records, validate schema and value correctness, and report errors by field and document type.

Continue learning

This article is part of the NLP and text analytics sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Compare a transformer with a linear baseline.
• Continue with Measure extraction precision and recall by field.

Reference: Python regular-expression documentation.