Message Queues
Decide when a queue earns its place, size a consumer fleet from arrival rate and job time, and handle the duplicates that at-least-once delivery forces.
A synchronous call couples two services' availability: if the callee is down, the caller fails with it, and if the callee is slow, the caller is slow. A message queue breaks that coupling by putting durable storage between them, so the producer's job is finished the moment the message is accepted.
The parts
A message queue is an intermediary component carrying asynchronous communication between parts of a system, usually separate services. It behaves like a mailbox: messages sit there durably until someone takes them.
- Producer (publisher) — creates and sends messages to the queue.
- Consumer (subscriber) — retrieves messages and processes them, at its own pace.
- Queue or topic — the named channel inside the broker. A message on a queue (point-to-point) is consumed by exactly one consumer; a message on a topic (publish/subscribe) reaches every subscriber.
- Message broker — the software running the channels: RabbitMQ, Apache Kafka, AWS SQS, Google Cloud Pub/Sub, Azure Service Bus. The choice changes the guarantees you get, and Kafka vs RabbitMQ is that comparison.
- Message — the unit of data: a JSON payload, a serialized object, a command, an event.
The producer sends and continues its own work without waiting; the consumer reads when it is ready.
What the queue buys
Decoupling. Producers and consumers need no knowledge of each other — no address, no API — only the channel. A UserService publishes a UserCreated event to a topic; NotificationService, AnalyticsService and WelcomeEmailService subscribe and react without it knowing they exist, and a fourth subscriber changes no producer code.
Responsiveness. Slow work leaves the request path: a web server takes an image upload, enqueues it and answers immediately, while a separate service resizes and watermarks later.
Load levelling. When production outruns consumption the queue is the buffer that keeps consumers from being flattened — a burst of sensor data accepted at full rate, analysed at a sustainable one.
Fault tolerance. A dead consumer costs latency, not data: messages stay in durable storage until it restarts. Acknowledgements finish that guarantee, since the broker holds a message until the consumer confirms it processed it, so a crash mid-processing means redelivery.
Scalability. Consumers scale independently of producers, and a growing queue is a capacity signal with an obvious response.
Sizing the workers, and what a spike costs
Take that image pipeline: 1M uploads/day is about 12/s average, peak is 3× at roughly 36/s, and each resize costs 2 s of CPU.
Done synchronously, Little's Law fixes the concurrency for you: 36/s × 2 s = 72 requests in flight at peak, each holding a thread and an open connection for the full 2 s. A 50-thread web tier is past its limit, and page loads fail alongside the uploads because they share those threads.
Behind a queue the producer's work is one durable write of a few milliseconds, and the 2 s moves to a fleet sized on its own: worker concurrency = throughput × job time, so 30 workers give 30 ÷ 2 = 15 messages/s. Size for the average plus a little headroom and the spike arrives 21/s faster than it drains:
backlog after a 30-min peak = (36 - 15) msg/s × 1,800 s = 37,800 messages
spare drain rate afterwards = 15 - 12 = 3 msg/s
time to clear = 37,800 ÷ 3 ≈ 12,600 s ≈ 3.5 hours
Nothing was dropped, which is what the queue is for, but someone who uploaded during the peak sees their thumbnail three and a half hours later. At 48 workers (24/s) the deficit is 12/s over 1,800 s = 21,600 messages, drained at 24 − 12 = 12/s spare, clearing in about 1,800 s — the length of the spike itself. Async does not remove latency, it moves it into queue depth, where nothing alerts on it unless you say so. Queue depth and consumer lag are the graphs to name.
Delivery guarantees
- At-most-once — a message may be lost but is never delivered twice. Acceptable for a metrics sample, not for a payment.
- At-least-once — a message is never lost but may arrive more than once. The common default.
- Exactly-once — delivered once and only once. Attractive, and mostly a boundary condition rather than a product feature.
At-least-once implies duplicates, and duplicates imply an idempotent consumer. The redelivery path is ordinary: a consumer processes a message, sends the email, and crashes before its ack reaches the broker; the broker redelivers and the customer gets a second email. Nothing malfunctioned. Key the side effect instead — write the message id under a unique constraint and check it before acting — which is the whole of idempotency.
Read an exactly-once claim narrowly. A broker deduplicates inside its own boundary with producer sequence numbers and transactional offsets; it cannot make an SMTP server or a payment API forget. The feature is at-least-once delivery plus deduplication that stops at the broker's edge.
What the queue costs
Poison messages. At-least-once plus a message that fails every time is an infinite redelivery loop holding a worker slot. Cap attempts — five is typical — and route the message to a dead-letter queue, then alert on DLQ depth; a dead-letter queue nobody watches is data loss with extra steps.
Ordering. One queue with eight consumers has no global order: message 2 can finish before message 1. Where order matters per entity, partition by key so every event for user 42 lands on one partition — order per key survives, parallelism caps at the partition count, and a hot key sets a ceiling no repartitioning removes.
The lost publish. Commit the order row, publish order.placed, and the broker times out: the order exists and the warehouse never hears. Two writes, no transaction across them — the dual-write problem, whose answer is the transactional outbox.
A queue is also the wrong tool when the caller needs the result to render its response. And below a few hundred jobs per second, the alternative worth naming out loud is a database table used as a queue with SELECT … FOR UPDATE SKIP LOCKED: one less system to run, and the enqueue joins the business transaction, so dual-write never arises. It stops being right when polling competes with the OLTP workload against the ~5,000 QPS a commodity Postgres box has, or when several independent subscribers need the same event.
In an interview
Message queues are a common pattern, and proposing one is the easy half. What is tested is whether you introduce asynchrony for a stated reason and then carry its consequences — the graded part is the follow-up: what happens when the consumer is down for an hour, and what happens when a message arrives twice.
Name what the queue buys in the terms of the problem on the board — decoupling producers from consumers, slow work off the request path, absorbing spikes, surviving a downstream outage — then close the loop in one breath: "this write path is 2 s of image work against a 200 ms budget, so the API enqueues and returns 202 with a status URL. Delivery is at-least-once, so the worker dedupes on message id before writing. Peak is 3× average and the fleet drains a 30-minute spike in about 30 minutes; queue depth is the alert."
The mistake that loses points is drawing a queue on the diagram and moving on — no delivery guarantee named, no idempotency requirement on the consumer, nothing about what the user sees while a backlog drains. Two smaller ones follow: taking a broker's exactly-once label as permission to skip deduplication, and queueing work whose result the caller is blocked on anyway.
Check yourself
1. A transcode queue averages 5 jobs/s, each job takes 40 s, and you run the fleet at 20% headroom. The consumers are down for 10 minutes. How long until the queue is empty again?
Keeping up needs 5/s × 40 s = 200 workers in flight; 20% headroom makes it 240, or 6 jobs/s. Ten minutes down accumulates 5/s × 600 s = 3,000 jobs, and draining uses only the spare 6 − 5 = 1 job/s, so 3,000 s ≈ 50 minutes of catch-up for 10 minutes of downtime. Headroom, not fleet size, sets recovery time — say that 5× multiplier out loud before promising a delivery SLO.
2. Your broker advertises exactly-once delivery and the consumer charges credit cards. Do you still build the dedupe table?
Yes. The guarantee covers the broker's own path, producer to log to committed offset; the charge goes to a third-party API outside that transaction, so a redelivery after a crash between charging and acking still bills twice. Treat it as at-least-once: key the charge on the order id with a unique constraint, so the second attempt violates it and returns the first result. One spare row against one double charge.
3. Payment events must apply in order per account. You have 12 consumers in the group and 4 partitions keyed by account id. What throughput do you get, and what changes?
Four partitions' worth. One consumer per partition per group leaves 8 of the 12 idle, and round-robin instead would lose per-account ordering. Raise the partition count to at least 12, keyed by account id: order per account survives and every consumer works — unless one account is hot, in which case its partition is the ceiling and that account needs batching or its own path.