AI Agent Orchestration: Retries, Timeouts and Circuit Breakers That Actually Help

Orchestration is a broad word. For some teams it means chaining several agents into a workflow. For operators it means something more basic: getting a request to the right agent, giving it enough time to finish, recovering when it does not, and keeping a trace of what happened. The second meaning is where most production incidents live, and it is the focus of this article.

Why agents are harder to orchestrate than APIs

• Latency is wide. A simple answer might take 400 milliseconds. A multi step tool chain might take 90 seconds.
• Failures are mixed. Some come from your code, some from the model provider, some from tools the agent calls.
• Calls are expensive. A retry is not free when every attempt spends tokens.
• Side effects are real. An agent that files a ticket or sends an email must not do it twice because a caller retried.

Pattern 1: Set timeouts at every layer, from the inside out

Timeouts should get longer as you move outward. The model call inside the agent has the shortest timeout, the agent's own request handler a little longer, the gateway in front of the agent longer again, and the client longest of all. If the order is reversed, the client gives up while the agent keeps working and spending.

In a beanshost.yaml manifest the agent level timeout is set under spec.resilience.timeoutSeconds, between 1 and 300 seconds. The Python example in the BeansHost console sets the client timeout to 330 seconds for exactly this reason.

Pattern 2: Retry less than you think

Retries help with brief, transient failures such as a dropped connection or a 503 from an overloaded provider. They hurt when the failure is deterministic, when the call has side effects, or when many clients retry at once and turn a slowdown into an outage. A few rules keep retries useful:

1. Retry only on errors that are likely to be transient: timeouts, 502, 503 and 429 with a retry hint.
2. Never retry a 4xx caused by the request itself, such as a 400 or 413.
3. Cap attempts. Three is plenty for most agents, and the manifest allows 1 to 10.
4. Use exponential backoff with jitter so clients do not retry in lockstep.
5. Make side effects idempotent, for example by passing a request ID the agent can check before acting.

async function callWithRetry(send, attempts = 3) {
  for (let attempt = 1; attempt <= attempts; attempt++) {
    const response = await send();
    const retryable = [502, 503, 504].includes(response.status) ||
      (response.status === 429 && attempt < attempts);
    if (!retryable || attempt === attempts) return response;
    const base = 2 ** attempt * 250;
    await new Promise((r) => setTimeout(r, base + Math.random() * base));
  }
}

Pattern 3: Put a circuit breaker in front of each agent

A circuit breaker watches for repeated failures. When they cross a threshold, it stops sending traffic for a short period and answers immediately instead. That protects the struggling agent from a pile of new requests and protects callers from waiting on timeouts that are almost certain to fail. After a cool down, a few trial requests decide whether to close the breaker again.

BeansHost applies a breaker in its runtime dispatch. When it is open, calls answer 503 runtime_circuit_open so your application can show a clear message or fall back without waiting. The full list of responses is in the agent API error reference.

Pattern 4: Give every run an identity

When something goes wrong, the first question is which run. Assign an execution ID to every call, accept or create a correlation ID that follows the request across services, and propagate a W3C traceparent header so tracing tools can stitch spans together. Return those identifiers to the caller so support can find the run from a customer report.

{
  "executionId": "0b6c2f7e-6d2a-4a0e-9d55-2f4f1c9e8a10",
  "correlationId": "req_8f2d1c",
  "status": "succeeded",
  "data": { "reply": "The agent's own response" }
}

Pattern 5: Distinguish failure types

A timeout, an unreachable agent and a quota limit all look like failures to a user, but they need different fixes. Make sure your orchestration layer returns stable codes rather than free text. A caller can then retry agent_timeout, alert on agent_unreachable and show an upgrade prompt for execution_quota_exceeded without parsing messages.

A short checklist

• Timeouts increase from the innermost call to the client.
• Retries are capped, jittered and limited to transient errors.
• Side effects are idempotent.
• A circuit breaker protects each agent.
• Every run has an execution ID, a correlation ID and trace context.
• Errors use stable codes your callers can act on.

Once these basics are in place, scaling becomes a matter of capacity rather than firefighting. Our guide to rate limits, quotas and concurrency picks up from here.