Scaling AI Agents: Concurrency, Rate Limits and Monthly Quotas

When an agent gets popular, the first instinct is to add instances. That helps, but it is rarely the first limit you hit. Agents spend most of their time waiting on model providers and tools, so the real ceilings are often concurrency per instance, the provider's own rate limits and the quotas you set to protect your budget. Planning for all of them together avoids surprises.

Three different limits

Concurrency: how much one instance can hold

Because agent runs are mostly I/O, a single instance can often handle dozens of concurrent runs. The limiting factor is usually memory, especially if the agent loads documents or keeps conversation state. Measure memory under realistic load and set concurrency below the point where it climbs sharply.

spec:
  scaling:
    minimumInstances: 1     # 0 to 300
    maximumInstances: 5     # 1 to 300
    concurrency: 50         # 1 to 1000 runs per instance

Setting minimumInstances above zero keeps one instance warm and removes cold starts for containers. Setting maximumInstances puts a ceiling on spend during a spike. How each value is applied depends on the provider you deploy to, so check the provider's own scaling settings as well.

Rate limits: the short window

Rate limits cap calls per minute. They protect shared infrastructure from a single runaway client and they protect you from bugs that call an agent in a loop. Calls through BeansHost are limited per organization by plan: 60 per minute on Builder, 600 on Team and 3,000 on Enterprise. Over the limit, calls answer 429 tenant_rate_limit.

Remember that your model provider has its own limits on requests and tokens per minute. Your agent's effective ceiling is the lower of the two, so check both before a launch.

Quotas: the long window

Monthly quotas cap total runs per period. They turn a surprise bill into a predictable error. On BeansHost, Builder includes 1,000 runs per month, Team 100,000 and Enterprise has no run cap. The period is the calendar month in UTC, and the console warns at 80 percent. Over the quota, calls answer 429 execution_quota_exceeded rather than adding charges.

Handling 429 responses well

1. Read the error code, not just the status. A rate limit is worth retrying shortly. A monthly quota is not.
2. Back off with jitter before retrying a rate limit.
3. Show users a clear message for quota errors and alert your team.
4. Never retry in a tight loop. It makes the limit last longer for everyone.

const result = await response.json();
if (response.status === 429) {
  if (result.code === "tenant_rate_limit") {
    await sleep(1000 + Math.random() * 1000);
    return retry();
  }
  if (result.code === "execution_quota_exceeded") {
    notifyTeam("Monthly agent runs used");
    throw new Error("Agent is unavailable until the quota resets");
  }
}

A capacity planning worksheet

Before a launch, run through a short calculation for each agent:

• Peak calls per minute. Expected peak traffic, multiplied by a safety factor of two.
• Average run time. Use p95 rather than the mean.
• Concurrent runs at peak. Calls per second multiplied by p95 duration in seconds.
• Instances needed. Concurrent runs divided by safe concurrency per instance.
• Monthly runs. Average daily calls multiplied by thirty, compared with your plan.

For example, 300 calls per minute at peak is 5 per second. With a p95 of 4 seconds, that is about 20 concurrent runs, comfortably handled by one instance at a concurrency of 50, with a second instance for headroom. 300 calls per minute fits inside the Team plan's 600 per minute rate limit.

Key takeaways

• Concurrency, rate limits and quotas are separate limits with separate fixes.
• Your effective ceiling includes the model provider's limits.
• Handle 429 by error code: retry rate limits, surface quota errors.
• Plan capacity from p95 duration and peak traffic, not averages.

Compare limits for every plan on the pricing page, and see the full list of API responses in the developer reference.