Networks and the cost of a round trip
Price a request in round trips (DNS, TCP, TLS and every internal hop) and use that number to choose protocols, regions and connection pool sizes.
Every remote call costs time that faster code cannot recover: the round trip. Inside one datacenter it is about 0.5 ms; from a phone in India to a server in US East it is about 200 ms. Those two numbers decide where services live, which protocol carries the traffic, and how many connections a pool has to hold.
Addressing: how a packet finds a machine
A network is a collection of interconnected devices — clients, servers, routers, switches — that can address and reach one another to share resources and information.
Every device has an IP address. IPv4 is 32 bits (192.168.1.1), about 4.3 billion addresses, and it ran out; IPv6 is 128 bits (2001:0db8:85a3:0000:0000:8a2e:0370:7334) and fixes that exhaustion. A subnet is a logical subdivision of an IP network. A switch moves frames between devices on one local network; a router forwards packets between networks, picking the next hop toward anything outside your subnet.
IPv4 exhaustion put most clients behind NAT, with a consequence candidates miss. A request crosses a carrier NAT and a load balancer before reaching an app server, so the source IP the server reads is the proxy, not the user. Per-IP rate limiting on that address throttles your own load balancer, or a whole carrier as one client. Use X-Forwarded-For, and trust it only from your own edge.
The four layers
The TCP/IP model splits communication into four layers, each wrapping the one above.
| Layer | Job | Examples |
|---|---|---|
| Application | Network services for applications | HTTP, DNS, SMTP, FTP |
| Transport | Delivery between processes on two hosts | TCP, UDP |
| Internet | Addressing and routing between networks | IP |
| Link | Bits over a physical medium | Ethernet, Wi-Fi |
DNS is the decentralised naming system turning google.com into an IP address; HTTP is the request-response protocol the web runs on, and HTTPS is HTTP with TLS underneath. The TCP/IP model lesson goes layer by layer.
What one request actually costs
Count the round trips on a cold request from India to an origin in US East, at roughly 200 ms each:
DNS lookup 1 RTT
TCP handshake 1 RTT
TLS handshake 1 RTT (TLS 1.3; TLS 1.2 costs 2)
The request 1 RTT
4 RTT x 200 ms = 800 ms before the first byte
None of that is server work, so tuning the application recovers none of it. Keep-alive drops the handshakes from later requests: request two costs 1 RTT, not 4. Terminating TLS at an edge 20 ms away cuts setup from 600 ms to about 60 ms, leaving one 200 ms trip to the origin — about 260 ms, or 80 ms when the edge answers from cache. That is the CDN argument as arithmetic rather than preference.
Inside a datacenter, 0.5 ms per hop reads as free until you count hops: twenty sequential internal calls spend 10 ms of network time before any service works, and p99 is worse because a tail on any hop lands in the total. A datacenter round trip is 500,000 ns against a 100 ns memory reference, 5,000 times more expensive — which is where microservices boundaries get decided: split only when the split buys more than the 0.5 ms it adds.
TCP or UDP, and what each gives up
TCP is connection-oriented and gives reliable, ordered delivery. UDP is connectionless and gives unreliable, unordered delivery with far less overhead.
The trade-off is clearest as a failure. Carry a voice call over TCP and lose one packet: TCP hands the application nothing queued behind it until the retransmit arrives, one more round trip, 200 ms here. The listener hears silence, then a burst of stale audio. That head-of-line blocking is why the obvious choice is wrong — a 20 ms frame delivered 200 ms late has no value, so retransmitting it is worse than dropping it, and UDP delivers what arrived while the codec conceals the gap.
Invert the case and TCP wins outright: where a missing byte corrupts the meaning — a payment, an HTML document, a database write — ordered, guaranteed delivery is what you are paying for. For a connection held open both ways, see WebSockets.
Where the network breaks a design
A slow remote dependency exhausts a connection pool. Little's Law: concurrency = arrival rate x latency. A service taking 800 QPS with a 10 ms dependency keeps 800 x 0.01 = 8 calls in flight. Move it cross-region at 210 ms and the same traffic needs 800 x 0.21 = 168. A pool of 100 makes requests queue; queueing raises latency, which raises required concurrency again, and the service saturates with its CPU idle. Resize the pool, or take the call off the synchronous path with a queue — neither option is visible until you do the multiplication.
DNS is a cache with a TTL, not a switch. Repointing a record takes effect only as resolvers expire the old entry. At a 300-second TTL and 1,000 QPS peak, that is up to 300,000 requests sent to a dead address after you believe you fixed it, and some clients hold records longer. Failover belongs behind a load balancer or an anycast address, with TTLs on movable records set to 60 seconds beforehand.
Bandwidth is a ceiling that arrives earlier than expected. A 1 Gbit/s NIC moves about 125 MB/s: on a 20 KB JSON response that saturates at 125 / 0.02 = 6,250 QPS, and on a 1 MB image at 125 QPS, long before CPU becomes the limit.
Topology sets failure domains: two replicas sharing one rack switch have a single switch between them and an outage. Security belongs in the same picture — private subnets, firewall rules, encryption in transit.
In an interview
Networking rarely gets its own question. It arrives as the follow-up: "the client is in Singapore and your database is in Virginia — what does that do to your p99?" What is tested is whether you price a design in round trips instead of adjectives.
Answer with the count. "Cold, that path is four round trips — DNS, TCP, TLS, then the request. At 200 ms each that is 800 ms before a byte of content, and none of it is server time. I would terminate TLS at an edge, hold connections open, and serve reads from a regional replica so only writes cross the ocean."
The mistake that loses points is answering a latency problem with bandwidth. More bandwidth does nothing for a 200 ms round trip: bandwidth is bytes per second once data flows, latency is distance and hop count. More lanes do not shorten the motorway. The mirror-image mistake costs as much — treating an internal call as free, then making 40 of them per request.
Check yourself
A mobile client in India talks to a single origin in US East, and the page needs six sequential API calls on a cold connection. Estimate time-to-usable, and name the change that helps most.
Setup is DNS + TCP + TLS = 3 RTT = 600 ms, and six sequential calls add 6 x 200 ms = 1,200 ms. Roughly 1.8 s, with no server time counted. The win is fewer round trips, not a faster backend: parallelise or collapse the six calls into one response (1,200 ms becomes 200 ms), then terminate TLS at an edge to cut setup to about 60 ms.
A service takes 800 QPS and calls a dependency whose latency went from 10 ms to 210 ms after a region move. Its pool holds 100 connections. What happens, and what size was needed?
It needs 800 x 0.21 = 168 concurrent connections against a pool of 100. Requests queue, latency climbs past 210 ms, required concurrency rises again, and the service saturates while CPU sits idle. About 200 would hold it; better, move the call off the synchronous path or back in-region — before the move it needed 800 x 0.01 = 8.
Choose a transport for a live sports stream at 3 Mbit/s to 50,000 concurrent viewers, and say what it costs you.
A UDP-based transport such as QUIC: a frame retransmitted after its playback deadline is worthless, and TCP head-of-line blocking stalls every frame behind a lost packet for a full round trip. You give up ordering and delivery guarantees, so the player conceals gaps. Then the bandwidth: 50,000 x 3 Mbit/s = 150 Gbit/s, about 150 saturated 1 Gbit NICs — a CDN fan-out problem before a server problem.