Few-shot examples: choose cases that expose the boundary

Examples teach more than output style. They imply which inputs are supported, when the system should abstain and how conflicts are resolved. Choosing only easy cases can make the prompt look successful while leaving its critical boundary unspecified.

Cover behaviours deliberately

The offline foundations lab defines four examples.

from foundation_cases import few_shot_case

result = few_shot_case()
assert result["coverage"] == 4
assert result["selected_by_similarity_only"] is False
assert set(result["covered_boundaries"]) == {
    "supported", "unsupported", "contradictory", "untrusted_instruction"
}

The set includes a normal supported case, missing evidence, contradictory evidence and an untrusted instruction in retrieved content. No model executes; the check verifies the example-selection policy.

Build an example inventory

For each candidate, record the task slice, intended behaviour, why it is difficult and which instruction it clarifies. Prefer a minimal set that covers distinct boundaries. Near-duplicate normal examples can consume context without teaching new behaviour.

Match the real input distribution, but preserve rare high-impact cases. An approval assistant might see missing documents infrequently, yet that case deserves explicit abstention behaviour. Include the exact structured output and evidence references expected from the application.

Keep examples separate from evaluation

Few-shot examples are part of the prompt and therefore training-like context. Do not count them as evaluation cases. Freeze a separate labelled set before comparing prompt versions. Check whether changing one example helps its slice while harming another.

Avoid copying sensitive customer content into prompts. Create representative redacted or synthetic examples with reviewed labels. Track provenance, rights and version. If examples are retrieved dynamically, test selection leakage and injection risks as part of the system.

The Generative & Agentic AI course connects few-shot design to context budgeting, prompt evaluation and adversarial testing.

Exercise

List ten candidate examples for a document decision. Tag their behavioural boundaries, select the smallest useful set and justify every inclusion. Evaluate against a separate fixed set, then remove each example in turn to measure its effect.

Continue learning

This article is part of the LLM fundamentals and prompt design sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Validate JSON output beyond syntactic correctness.
• Continue with Evaluate prompt changes on a fixed task set.

Reference: OpenAI prompt-engineering guidance.