Webhooks versus polling
Choose webhooks, polling or both for a connector, price a poll in requests against the rate limit, and verify a webhook before acting on it.
On this page
A connector learns that something changed in one of two ways: the other system calls it, which is a webhook, or it asks, which is polling. Webhooks arrive in seconds and cost nothing when nothing happens, but they are delivered at least once, in no guaranteed order, and occasionally not at all. Polling spends requests whether or not anything changed, but the connector chooses when, and nothing the API still holds can be missed. Connectors that move money use both.
Polling, priced
The agency's CRM holds 50,000 contacts and allows 60 requests a minute; its list endpoint returns 100 contacts a page.
full sync = 50,000 ÷ 100 = 500 requests
time = 500 ÷ 60 a minute ≈ 8.3 minutes
The first poller anyone writes re-reads everything on a timer. Every 15 minutes is 96 syncs a day, 48,000 requests, 56% of the 86,400 a day the limit allows, nearly all of them re-reading contacts that did not change. For 8.3 minutes of every 15 the poller holds the whole limit, and anything else on the same account, such as the lead workflow's CRM lookups, gets refused.
The fix is an incremental poll: ask only for records changed since a cursor, the updated_at of the last change seen. Assume 200 contacts change on a typical day. Polling every 5 minutes is 288 polls a day, almost all returning one page or nothing: about 288 requests, 0.3% of the limit, and a change is seen within 5 minutes, 2.5 on average.
Incremental polling has two blind spots. A deleted contact has no updated_at to find, so it never appears in a changed-since query. And a change stamped in the same second as the cursor, or on a server whose clock runs a few seconds behind yours, can fall between two polls. The fixes are an overlap and a sweep: start each poll's window a minute before the cursor and drop what you already have, and once a night run the 500-request full read to find deletions by comparing ids.
What a webhook promises
The provider sends an HTTP POST to your URL when something changes. Any 2xx response counts as delivered. Anything else, or no answer within the provider's timeout, which ranges from a few seconds to about 30 depending on the provider, and it resends, usually with backoff over hours or days, before giving up. Three properties follow, and each needs a decision:
- At least once. A resend after a timeout delivers an event you may already have processed. At-least-once delivery means duplicates, and duplicates mean the receiver must be idempotent: record each event's id and drop repeats. The derivation is idempotency, and why at-least-once forces it; the agency's version, with invoices, is idempotency and deduplication.
- Not in order. Resends and parallel deliveries mean
contact.updatedcan arrive beforecontact.created, or Tuesday's change after Wednesday's. Apply an event only if its version or timestamp is newer than what you hold, or treat every event as a notice that a record changed and fetch its current state, which makes order irrelevant at one request per event. Where events carry a sequence number, a webhook deduper applies them in order. - Not always. If your endpoint is down for longer than the provider retries, the events are dropped. Some providers let you list or replay past events; the nightly sweep catches the rest.
Answer first, work later
The webhook handler has three jobs: verify the event, store it, return 200, all in well under a second. The work happens afterwards, from a message queue: finding the contact, calling a model, writing to the accounting system.
The usual first version does it the other way round, with all the work inside the handler before it replies. A model call of 6 to 9 seconds plus two API calls pushes the slow deliveries past a 10-second sender timeout. The sender resends events that were already processed, and duplicates arrive at exactly the rate the handler is slow.
Signatures: who sent this?
A webhook URL is an open door. Anyone who learns it can POST a convincing payment.received. Providers sign each delivery with an HMAC (RFC 2104 (opens in a new tab)), usually SHA-256, computed over the raw request body with a secret shared when the webhook was set up, and send it in a header with a timestamp. The receiver:
- recomputes the HMAC over the raw bytes it received, before parsing, because parsed and re-serialised JSON changes whitespace and key order and no longer matches;
- compares the two values in constant time;
- rejects a timestamp older than about five minutes, so a captured delivery cannot be replayed next week;
- keeps the secret where the OAuth tokens live, as the previous lesson sets out, not in a workflow step.
Header names and the exact string signed differ by provider, so read their webhook page; the shape above does not. The usual failure is step 1: a framework upgrade parses the body before the check, every signature fails, and someone turns verification off "until Monday".
Choosing
| Situation | Use |
|---|---|
| Webhooks exist and minutes matter | webhooks, plus a nightly reconciliation poll |
| No webhooks | an incremental poll with an overlap, plus a nightly sweep for deletes |
| Most records change every day | a poll; webhooks would send thousands of events for one sync's worth of data |
| Money moves on the event | both, and the reconciliation is not optional |
Webhooks for speed, a poll for completeness. Neither alone is a connector you can leave running unwatched. The poll's request count also has to fit the limit alongside everything else, which is rate limits, retries and backoff.
In practice
Ask the vendor four things. Is delivery at least once, and does the event id stay the same across resends? How long do you resend, and can we replay missed events? How are deliveries signed? Does the list endpoint filter by updated_at, and how do deletes show up?
The mistake that costs money is trusting webhooks alone. The agency's receiver is down for a weekend deploy; the payment provider resends for 24 hours and then drops the rest. Five payment.received events are lost, and on Monday the overdue-reminder automation chases five clients who have paid. At the $100 that the scoring frame puts on a reminder to a client who has paid, that is $500, and a nightly reconciliation poll of the provider's payment list, a request or two, would have caught every one.
Check yourself
1. The accounting system has no webhooks: 12,000 records, 50 a page, 30 requests a minute. The overdue-reminder automation must see a payment within 30 minutes. Full sync or incremental, and how often?
A full sync is 240 requests, 8 minutes at the limit. Every 30 minutes that is 48 syncs, 11,520 requests a day, and the whole limit for 8 minutes in every 30. An incremental poll every 10 minutes is 144 requests a day and sees a payment within 10 minutes. Incremental, plus the 240-request sweep once a night for deletes.
2. Two contact.updated events for one contact arrive 40 seconds apart; the second carries the older updated_at. Which do you apply?
The newer
updated_at, whichever arrived last. Last arrival wins is the bug: it writes Tuesday's address over Wednesday's. If the event carries no version or timestamp, fetch the contact's current state instead of applying either.
3. After a framework upgrade every webhook fails signature verification. The contractor proposes switching verification off until Monday. Decide.
Leave it on and keep returning an error. The provider resends what was refused, so nothing is lost while the fix is made inside its retry window. The fix is to verify the raw bytes before the framework parses them. Switching it off would leave the URL accepting any POST anyone sends.