Build an offline fixture for an LLM integration

Most integration logic does not require a live model call. Parsing, schema validation, state transitions, retry limits and downstream permissions can run against recorded or authored fixtures. This makes tests fast, deterministic and safe from accidental spend or side effects.

Define the interface fixture

The offline foundations lab creates one authored request/response pair.

from foundation_cases import offline_fixture_case

result = offline_fixture_case()
assert result["complete"] is True
assert result["network_called"] is False
assert result["suitable_for_contract_test"] is True
print(result["fixture"])

The record contains request, response, status and headers. Its header marks offline-v1, and the response abstains for a no-receipt case. It is not a captured provider result.

Test the adapter boundary

Wrap the provider SDK behind a small interface owned by the application. Feed fixtures through the same response parser and validator used in production. Include stable metadata the application relies on, while avoiding a brittle copy of every provider field.

Create fixtures for success, refusal, timeout, rate limit, truncated stream, invalid JSON, unknown enum and tool-call request. Verify state changes, maximum retries, user-visible status and that side effects remain disabled without validated approval.

Separate three test levels

Unit tests use authored objects and target application logic. Contract tests verify that the adapter understands a provider-shaped response. A small, controlled live test verifies current credentials and API compatibility. Only live eval runs can measure model behaviour, latency and current cost.

Sanitize recorded fixtures before committing them. Remove prompts, user data, keys, request identifiers and content that policy forbids retaining. Record provenance and terms for any retained sample.

The Generative & Agentic AI course uses offline fixtures to make integrations reproducible before learners add providers, agents or deployment.

Exercise

Implement an adapter and six fixtures. Prove tests run with networking disabled. Add one controlled live compatibility check separately, and document which claims each layer can and cannot support.

Continue learning

This article is part of the LLM fundamentals and prompt design sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Compare models using task quality, latency and cost.
• Continue with Prompt injection versus an ordinary user instruction.

Reference: Python `unittest.mock` documentation.