WebSockets
Price polling against a persistent connection, keep long-lived sockets alive past idle-timeout reapers, and say what the stateful connection tier costs.
HTTP's request-response shape has one limitation that changes architectures: the server cannot speak first. Every design needing server-initiated updates — chat, live dashboards, collaborative editing — either works around that with polling or removes the constraint with WebSockets.
The problem with HTTP for real-time
HTTP is request-response: the client sends a request, and the server sends a response. This works well for fetching web pages or data on demand, but it is less suited to the case where the server needs to push information to the client — a new chat message, a stock price update, someone else's cursor moving in a shared document.
Workarounds exist. Polling has the client repeatedly ask "any updates?". Long-polling has the client ask once and the server hold the request open until an update arrives or a timeout fires. Both imitate push badly: polling wastes requests and still leaves latency on the table, and long-polling holds a connection open anyway — the cost people were trying to avoid.
HTTP/2 and HTTP/3 do not fix it either. They multiplex streams over one connection and compress headers, but a stream still starts with a client request, and HTTP/2 server push — designed for pre-loading assets, not application events — has been removed from browsers.
Pricing polling against a socket
Take a live dashboard with 500,000 concurrent viewers, polling every 2 seconds.
Poll rate = 500,000 ÷ 2 s = 250,000 requests/second
Bytes on air = 250,000 × ~1 KB ≈ 250 MB/s
Real events = 20,000/second (8% of polls carry anything)
Latency = 0 to 2,000 ms, ~1,000 ms average
The ~1 KB is cookies and headers up, an empty JSON response and its headers back — the payload is the small part. Tightening the interval does not help: halving it to one second doubles the rate to 500,000 requests/second and still leaves 500 ms of average delay. Cost scales linearly with polling frequency; the latency floor is whatever interval you can afford.
The same load over WebSockets carries only the 20,000 real events at roughly 200 bytes each — about 4 MB/s, some 60× less traffic, and the delay collapses to a frame on an already-open socket: the 0.5 ms datacenter round trip plus the client's last mile, rather than a full connection setup.
What the socket costs instead is memory that never goes away. Budget ~10 KB per connection for kernel socket buffers plus per-session state, and around 100,000 connections per gateway node: 500,000 × 10 KB = 5 GB of connection state, five nodes standing by whether anything is happening or not. Polling burns CPU and bandwidth in proportion to traffic; WebSockets burn memory and file descriptors in proportion to users.
The upgrade handshake
A WebSocket connection starts as HTTP and stops being HTTP. The client sends an ordinary GET with Upgrade: websocket, Connection: Upgrade, a randomly generated Sec-WebSocket-Key, and Sec-WebSocket-Version. A server that understands the request answers 101 Switching Protocols with matching Upgrade and Connection headers plus Sec-WebSocket-Accept.
The accept value is worth one level down. The server takes the client's key, appends the fixed string 258EAFA5-E914-47DA-95CA-C5AB0DC85B11, hashes it with SHA-1 and base64-encodes the result. This is not security — the GUID is published in RFC 6455. It proves the responder parsed the upgrade, rather than being a cache or proxy replaying a canned 200 OK at a request it never read.
After the 101 the same TCP connection carries frames both ways. A frame header is 2 bytes minimum, plus 2 or 8 for extended lengths, plus a 4-byte masking key on anything a browser sends — masking exists so a hostile page cannot shape bytes a naive proxy mistakes for a fresh HTTP request and caches. Set 2 to 14 bytes of framing against the several hundred bytes of headers on every HTTP request and "low overhead" becomes a number rather than an adjective.
Run wss:// in production. TLS is why the upgrade survives intermediaries that would otherwise inspect and reject it, and it is the only version an HTTPS page can open.
The connection is state
An open socket lives in one process's memory on one machine. That single fact drives every operational difference from a REST tier.
A stateless app server scales and restarts freely because any node can serve any request. A WebSocket gateway cannot: delivering to a specific user means reaching the one node holding that user's socket. You need a registry mapping user to node, hit once per delivery, or a pub/sub fanout that broadcasts to all gateways and lets each drop what it does not own. The chat design works that registry through end to end.
The load balancer has to proxy the upgrade rather than terminate it, and its idle timeout must exceed your heartbeat interval. Balance by least-connections, not round-robin: connections are long-lived, so a freshly restarted node under round-robin receives only new arrivals and stays nearly empty while its peers stay full.
The failure mode nobody plans for
A healthy WebSocket carrying no traffic is indistinguishable from a dead one to every box in the path. Load balancers, corporate proxies and NAT devices reap idle connections — 60 seconds is a common default, and some are shorter. The socket closes, nobody tells the client, and the page sits there looking connected while nothing arrives. TCP keepalive does not rescue you: Linux waits two hours before the first probe.
The fix is application-level ping/pong frames on a schedule under the shortest timeout in the path, with the client treating consecutive missed pongs as death and reconnecting with jittered backoff. The jitter is not decoration. Restarting one node drops 100,000 connections at once; if every client retries immediately that is 100,000 handshakes per second, each carrying a TLS negotiation, aimed at a fleet already at capacity. Spread over 60 seconds it is ~1,700 handshakes/second, which the remaining nodes absorb.
The second failure is backpressure. A client on a bad mobile link stops draining and the server's per-connection write buffer grows; unbounded, 100,000 slow clients is how a gateway runs out of memory. Bound the buffer, drop the connection when it fills, and let the client reconnect and resync from a sequence number.
When to use them, and when not
WebSockets suit chat, multiplayer game input, collaborative editing and market data feeds — anything where the client also sends at a meaningful rate, or where binary frames matter.
The alternative worth rejecting out loud is Server-Sent Events. Where the flow runs one way — notification feeds, live scores, build logs, a progress bar — SSE gives push over ordinary HTTP, with browser-managed reconnection and Last-Event-ID resumption you would otherwise write yourself. It needs no upgrade, passes anything that speaks HTTP, and costs a connection just the same. The counter-argument is client chatter: typing indicators and cursor positions over SSE mean a second channel of POSTs, and at that point one socket is simpler.
In an interview
What is being tested is whether you can tell push from polling on numbers, and whether you notice that persistent connections make a tier stateful. Interviewers reach for WebSockets in chat, notifications and collaborative-editing prompts precisely because the stateful consequence is where candidates stall.
Say it in this order. Price the alternative: "polling 500,000 viewers every 2 seconds is 250,000 requests/second, 92% of them empty, and still a second of average latency — so I will hold connections instead." Name the cost in the same breath: "which makes the gateway tier stateful, so I need a user-to-node registry hit once per delivery, the balancer proxying the upgrade, and drained deploys." Then bound it: 10 KB per connection, ~100,000 per node, so 10M concurrent connections is roughly 100 gateways. Finish with the rejected option — SSE if traffic only flows one way.
The mistake that loses points is choosing WebSockets on vibes. "It's real-time" is a preference, not an argument, and the follow-up question is always what it costs. The second-worst answer treats a WebSocket tier like a stateless one and says nodes can be restarted freely; the interviewer will ask what happens on deploy, and the answer is 100,000 dropped sockets reconnecting at once unless you jittered them.
Check yourself
1. A dashboard has 200,000 concurrent viewers and each sees an update roughly once every 10 minutes. Polling every 5 seconds, or a persistent connection — and which kind?
Polling costs 200,000 ÷ 5 = 40,000 requests/second to deliver 200,000 ÷ 600 ≈ 330 updates/second, so 99.2% of requests return nothing — push wins on that ratio alone. But the traffic runs one way, so the answer is Server-Sent Events, not WebSockets: same single connection, same push, plus browser-managed reconnection and no upgrade for intermediaries to mishandle. Reach for WebSockets once the viewer starts sending too.
2. Your gateways sit behind an L7 load balancer with a 60-second idle timeout, and some users are behind a corporate proxy that reaps at 45 seconds. Pick a heartbeat interval and a death rule.
Size against the shortest reaper, not the one you control. Ping every 20 seconds so two beats fall inside every 45-second window and one lost beat does not cost the connection; declare the socket dead after two consecutive missed pongs, about 40 seconds. Do not rely on TCP keepalive — the Linux default waits two hours before probing, so the user stares at a live-looking page for the rest of the session.
3. You run 10M concurrent connections at ~100,000 per node. A rolling deploy restarts one node. Estimate the reconnect load, and what changes if clients retry immediately.
10M ÷ 100,000 = 100 nodes, so one restart drops 100,000 connections — 1% of the fleet. Immediate retry means ~100,000 handshakes/second, each with a TLS negotiation, hitting 99 nodes already carrying 9.9M sockets, and the registry takes 100,000 deletes plus 100,000 writes in the same instant. Jittered backoff over 60 seconds turns that into ~1,700 handshakes/second, about 17 per node per second. The deploy is unchanged; only the client's retry policy decides whether it is a blip or an outage.