Glossary · Rate limitingAlso called throttling, 429, quotaChecked 27 September 2026
What is rate limiting, and what happens when your system hits one?
Rate limiting is a provider capping how many requests you may send in a window. Past the cap it refuses, usually with a 429. What happens next is a design decision: a system either queues, retries, moves to another key, falls back to another provider, or drops the work and reports success anyway.
In one lineThe provider says no. What happens next is your architecture.
Why it matters to you
Almost every AI system meets this, because model providers price on it. The common response is to upgrade the plan, which works until the next ceiling and leaves the behaviour unchanged.
The structural answer is a pool with a policy. Mooney runs three Groq keys capped at 28 requests a minute each against a provider ceiling of 30, with a rolling 60 second window per key, a 60 second cooldown on any 429, 5xx or throw, round-robin to the least loaded healthy key, and a log line per hop naming the key and the reason it moved. Two Deepgram keys carry roughly 1.4 million voice turns a month, which was audited into a measured headroom figure of 1,500 to 2,000 daily active users.
The log line is the part people skip and the part that pays. Without it, a system that silently degrades to its slowest provider looks identical to one that is fine.
If a proposal used this word and did not explain it, ask us.
The first conversation is with the people who would build it. If there is a fit, the Consult comes next.