Rate limits, retries and backoff
Pace a connector under a provider's rate limit, honour 429 and Retry-After, back off with jitter inside a retry budget, and never retry a 400.
On this page
Every SaaS API caps how fast you may call it, and connector failures cluster at the cap: a backfill that eats the limit, a burst that turns into thousands of refused calls, a loop that repeats a request the server has already rejected. A limit is a budget to plan against before the first call, not an error to retry through afterwards. Retries are a second budget, because each one bets that the same request will now succeed, and some of those bets are never worth placing.
The bucket the provider runs
Most providers enforce limits with something shaped like a token bucket, the algorithm rate limiting derives: a bucket holds up to B tokens and refills at r per second, every request takes one, and an empty bucket answers 429. B is the burst you may send at once; r is the rate you may sustain. The agency's accounting system allows 30 requests a minute, so r = 0.5 a second; assume a burst of 30. Read what the limit is counted per: an access token, an app, or the whole account. A per-account limit is shared by every integration and by staff working in any screen backed by the same API.
The Friday invoice run makes three calls per invoice: find the client, create the invoice, send it. Forty invoices is 120 calls. Fired all at once:
first 30 calls use the burst, succeed at t = 0
remaining 90 need 90 ÷ 0.5 a second = 180 s of refill
Three minutes, whatever the client does. What the client controls is how many refused calls it makes in those three minutes. If the 90 waiting calls each retry every 2 seconds, one gets through per round, since the bucket refills one token every 2 seconds, and the rest get 429s: 89 + 88 + … + 1 ≈ 4,000 refused requests to send 90. Some providers count refused calls against a daily quota, and some block a client that keeps hammering. A client-side limiter that releases calls at 0.5 a second sends the same 120 in the same three minutes with no 429s at all. Pacing yourself costs nothing; not pacing costs 4,000 requests. The limiter is a few dozen lines, written and tested in a token-bucket rate limiter.
429 and Retry-After
HTTP 429 Too Many Requests (RFC 6585 (opens in a new tab)) is the provider saying the bucket is empty. It often carries a Retry-After header (RFC 9110 (opens in a new tab)), either a number of seconds or a date: the provider's own estimate of when capacity returns. Wait at least that long, taking the larger of it and your own backoff. Many APIs also report remaining quota on every successful call, under header names that vary by provider; a connector that reads them slows down before it reaches zero instead of after.
A 429 that keeps coming back is not transient. The workload needs more than the limit, and the fix is a schedule, such as running the backfill overnight as backfills and building your own connector does, or a higher tier. More retries do not fix it.
Jittered exponential backoff
A 5xx or a timeout usually means overload, and a retry adds load. Wait base × 2^k before retry k, capped, giving 1, 2, 4, 8 seconds, and then take a random value between zero and that wait, so that 20 workers that failed together do not retry together. The mechanism, and why the fixed interval is the mistake, is in retries, timeouts and rate limits. For a connector, the numbers that matter are the cap on attempts, five is common, and the longest wait before an item goes to a dead-letter queue and a person: 1 + 2 + 4 + 8 = 15 seconds for the schedule above.
The retry budget
A cap per request is not enough. If the accounting API goes down completely, every call makes five attempts: five times the load on a system that is already failing, and five times your own share of the limit once it recovers. A retry budget caps retries as a share of traffic, say no more than 10% of requests in any minute. At 30 requests a minute that is 3 retries a minute. During an outage the connector sends 1.1 times its normal calls rather than 5 times, and everything past the budget fails fast to the dead-letter queue. A circuit breaker goes one step further, stopping calls entirely after a run of failures and sending one probe at intervals, so the connector stops making an outage worse.
What never retries
A retry sends the same bytes again. That helps only when the failure was in the server's state, not in the request:
| Response | Action |
|---|---|
| 2xx | done |
| 429 | wait for Retry-After, then retry |
| 500, 502, 503, 504, timeout | back off with jitter, if the call is a read or carries an idempotency key |
| 401 | refresh the token once; if it fails again, stop and alarm |
| 400, 403, 404, 409, 422 | never; dead-letter with the response body |
The table, the jitter and the budget together are one small function, built in a retry policy. Two rows carry the money. A 400 or 422 says the request is wrong: an invoice for a client with no tax number, a date in the wrong format. The same bytes fail the same way, so five attempts produce five 422s and a 15-second delay before anyone sees the error. The other is a write with no key that timed out. POST /invoices may have created the invoice before the connection dropped, and resending it blindly creates a second. Either send an idempotency key the API honours, or look the invoice up by its reference before deciding to resend. Both are the subject of idempotency and deduplication.
In practice
Ask the vendor: what is the limit, counted per what, and is it shared with your own screens? Do you send Retry-After and remaining-quota headers? Do refused requests count against a daily quota? Do write endpoints accept an idempotency key? For a connector you buy, ask to see its retry policy: which statuses it retries, how many times, and where the failures go.
The mistake that costs money is a retry loop with no status check. It retries the 422 on every run, burns the shared limit and delays every other automation on the account. On a timed-out POST it creates the duplicate invoice that the operations manager then has to void, and that the founder has to apologise for.
Check yourself
1. The CRM allows 60 requests a minute per account. The nightly sweep needs 500 requests; the lead workflow needs up to 10 a minute at any hour. What rate do you give the sweep, and how long does it take?
Leave the lead workflow its 10 and keep 5 for retries: 45 a minute for the sweep, 500 ÷ 45 ≈ 11 minutes. Taking all 60 finishes in 8.3 minutes and refuses lead lookups for those 8 minutes. Three extra minutes at night is the cheaper side of that trade.
2. POST /invoices timed out after 10 seconds. The API accepts no idempotency key, but every invoice carries a reference field set to the timesheet id. Retry?
Not blindly. Search for an invoice with that reference: if it exists, record it and move on; if not, create it. One extra request per timeout, against a duplicate that costs a void, a credit note and a client's trust. Run one worker per timesheet so two workers cannot both find nothing and both create.
3. The accounting API is down for two hours on Friday afternoon. The invoice run, 120 calls, restarts every 15 minutes until it succeeds, and each call makes five attempts. Estimate the calls sent into the outage, and what a 10% retry budget changes.
Eight restarts × 120 calls × 5 attempts = 4,800 calls against a dead API. With the budget, each restart sends its 120 calls plus about 12 retries, so 8 × 132 ≈ 1,060. With a circuit breaker, one probe a minute: about 120 in two hours, and the run resumes when the first probe succeeds.