Set retry and timeout budgets across agent steps

Three steps with three retries and long timeouts can exceed a user’s deadline even when every component follows its own default. Start from the end-to-end budget and allocate within it.

Verify the total

The agent controls lab records two synthetic steps.

from agent_cases import budget_case

result = budget_case()
assert result["total_time_budget_s"] == 7
assert result["used_s"] == 6
assert result["within_total"] is True
assert result["attempt_budgets_respected"] is True

Lookup uses two attempts and three seconds; draft uses one and three. The check is arithmetic, not a network load test.

Retry only eligible failures

Rate limits, temporary timeouts and selected server errors may be retryable. Invalid arguments, permission denial and policy refusal usually are not. Use bounded exponential backoff with jitter while checking remaining total time before each attempt.

Pass deadlines/cancellation to downstream calls. Reserve time for cleanup and a useful final response. Count model/tool calls and cost alongside seconds; a fast loop can still exhaust spend.

Test worst paths

Simulate repeated timeouts, slow success, rate-limit headers and uncertain write outcomes. Verify terminal status, attempts and no retry beyond approval expiry. Report p95 end-to-end latency from real execution separately from configured limits.

The Generative & Agentic AI course connects budgets to termination, error taxonomy and observability.

Exercise

Allocate a 20-second workflow across four steps. Inject every retryable/permanent error and prove the final deadline and cost cap hold under the worst permitted path.

Continue learning

This article is part of the Agent workflows and state sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Idempotency keys for agent tool calls.
• Continue with Human approval gates: what happens while approval is pending.

Reference: AWS retry with backoff guidance.