Temperature is not a reliability control

Lower temperature can reduce variation in sampling, but it cannot make missing evidence appear, fix a flawed prompt, enforce a business rule or guarantee identical output across model versions and infrastructure. A confidently repeated unsupported answer is still wrong.

Inspect the claim with a fixture

The offline foundations lab stores three authored outputs.

from foundation_cases import temperature_case

result = temperature_case()
assert result["temperature_zero_repeat_equal"] is True
assert result["unsupported_at_zero"] is True
assert result["reliability_requires_evaluation"] is True
print(result["scope"])

Two temperature-zero rows repeat {"decision":"approve"}, yet the fixture marks that decision unsupported. A higher-temperature row abstains correctly. These are constructed examples—not model runs—and they demonstrate why repeatability and validity are different axes.

Define reliability as observable requirements

For a policy assistant, reliability may require:

• valid structured output;
• evidence identifiers that exist in the supplied context;
• no approval without the required receipt;
• abstention when evidence is missing or contradictory;
• stable behaviour across normal, boundary and adversarial cases;
• acceptable latency and cost under the selected configuration.

Turn each requirement into a deterministic check or labelled evaluation. Run repeated trials when sampling is enabled and report pass-rate distributions, not a single convenient response. Pin the model snapshot and all decoding settings where the provider supports it, while expecting future migrations to require reevaluation.

Use configuration for the right purpose

Temperature can be a task parameter. A constrained extraction task may favour low variation; brainstorming may accept greater diversity. Choose it on a fixed set and record the trade-off. Schema-constrained decoding can improve format adherence, but semantic rules still need validation.

Operational reliability also needs timeouts, bounded retries, refusal handling and fallbacks. Retrying the same unsupported request at temperature zero can reproduce the same failure and increase cost.

The Generative & Agentic AI course treats sampling settings as part of a versioned configuration and evaluates task behaviour separately.

Exercise

Define five business invariants for one LLM task. Evaluate repeated runs at two temperatures on the same labelled cases. Report exact configuration, per-invariant pass rates and any response that is repeatable but wrong.

Continue learning

This article is part of the LLM fundamentals and prompt design sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Tokens and context windows: budget a real document task.
• Continue with Prompt instructions versus retrieved evidence.

Reference: OpenAI API parameter reference.