Catalog My learning Blog
Scalability Patterns: Load Balancing, Caching, Sharding · Phase 1 · Lesson 1 · 9 min read

How Load Balancing Works: DNS, L4 and L7

On this page
  1. Learning Objective
  2. Concept Explanation
  3. Worked Example: Tracing One Request Through a Three-Tier Balancing Path
  4. Real-World Examples
  5. Review Questions & Answers

Learning Objective

Explain the three distinct layers at which traffic gets distributed in a production request path, DNS, transport-layer (L4), and application-layer (L7), what each layer can and cannot see about a request, and why real systems almost always use more than one of them at once rather than picking a single favorite.

Concept Explanation

A load balancer exists to solve two problems that are related but genuinely separate. The first is capacity: one machine has a finite amount of CPU, memory, and network bandwidth, and past some request rate the only way forward is to put more machines behind a single address. The second is availability: once there is more than one machine, a single machine failing should be a non-event rather than an outage, which requires something in front that knows which machines are currently healthy. Teams often think of load balancing purely as the first problem, but in practice the availability half is what turns a load balancer from an optimization into a hard requirement, and it is the reason the health checking covered later in this phase matters as much as the distribution algorithm itself.

The first place distribution happens is DNS. When a client resolves api.example.com, the authoritative nameserver can return several A records, or a different record depending on where the query came from, and the client picks one. This is enormously cheap, since it costs the serving infrastructure nothing at request time, and it is the only mechanism that can steer a client to a different continent before a single packet of application traffic is sent. Its weaknesses are equally structural: DNS responses are cached by resolvers and by the client operating system for the duration of the TTL, and many clients ignore TTLs they find inconveniently short, so removing a failed server from rotation by editing DNS can take minutes to hours to actually take effect. DNS also knows nothing about how loaded any given server is, because it never sees the request at all.

The second place is layer 4, the transport layer. An L4 load balancer makes its decision from the TCP or UDP header alone: source IP, source port, destination IP, destination port, protocol. It does not terminate TLS, does not parse HTTP, and often does not even maintain a full connection to the backend, instead rewriting packet headers and letting the backend respond directly to the client in a technique called direct server return. Because it touches so little, an L4 balancer is extremely fast and can push enormous packet rates on modest hardware. The cost of that speed is blindness: it cannot route on a URL path, cannot read a cookie, cannot retry an individual failed HTTP request, and it sees a single long-lived connection carrying a thousand multiplexed HTTP/2 requests as exactly one unit of load.

The third place is layer 7, the application layer. An L7 load balancer terminates the connection, typically terminates TLS, parses the HTTP request, and can therefore make decisions on anything in it: route /api/v2/* to one backend pool and /static/* to another, pin a session by cookie, rewrite headers, enforce rate limits per API key, retry an idempotent request against a different backend when the first one returns a 503, and count load in units of actual requests rather than connections. The cost is that it must fully handle every request, which means more CPU per unit of traffic and a per-request latency cost measured in a small number of milliseconds. It is also the layer where TLS termination naturally happens, which centralizes certificate management but also means the balancer becomes a security-relevant component.

The practical answer in almost every real architecture is not to choose but to layer them. DNS, often backed by anycast, steers a client to a nearby region. Within that region an L4 tier absorbs the raw packet volume and spreads connections across a fleet of L7 proxies. The L7 tier then makes the smart, content-aware routing decisions and talks to the actual application servers. Each layer handles the decisions it is structurally capable of making cheaply, and the layering is why a request in a large system commonly crosses three or four load balancing decisions before any application code runs.

LayerDecides Based OnStrengthsStructural Limits
DNSResolver location, record set, weightsFree at request time, can steer across continents before any packet is sentTTL caching delays failover by minutes or more; no view of server load
L4 (transport)IP addresses, ports, protocolVery high packet throughput, low latency, supports direct server returnCannot see URLs, headers, or cookies; one connection looks like one unit of load
L7 (application)URL path, headers, cookies, method, body sizeContent-based routing, per-request retries, rate limiting, TLS terminationMust fully process every request; higher CPU cost and a few milliseconds of latency

Worked Example: Tracing One Request Through a Three-Tier Balancing Path

It helps to write out what actually happens to a single request in a layered setup, because the sequence makes clear which decision each tier is capable of making and which it simply cannot. Notice how the units of load being counted change at every hop: DNS counts nothing, the L4 tier counts connections, and the L7 tier counts requests.

Code
1. Client resolves api.example.com
   DNS (geo-aware) -> returns 203.0.113.10, the anycast VIP for the eu-west region
   Decision input: resolver subnet only. TTL 60s.

2. Client opens TCP + TLS to 203.0.113.10
   L4 tier hashes (src_ip, src_port, dst_ip, dst_port) -> proxy-07
   Decision input: packet headers only. The whole connection is pinned
   to proxy-07 for its lifetime, so connection-level stickiness is free
   but request-level balance is not.

3. proxy-07 terminates TLS, parses:
     GET /api/v2/orders/8821 HTTP/2
     Authorization: Bearer ...
   L7 routing table:
     /static/*   -> cdn-origin pool
     /api/v2/*   -> orders-svc pool   <-- match
     /*          -> web pool
   Picks orders-svc-12 by least-outstanding-requests.

4. orders-svc-12 returns 503 (it is shedding load).
   Only the L7 tier can act on this: the request is idempotent (GET),
   so proxy-07 retries it on orders-svc-04 and returns 200 to the client.
   The L4 tier below could not have done this - it never saw a "request"
   at all, only bytes on a connection.
A single request crossing three distribution layers, DNS choosing a region, an L4 tier pinning a TCP connection to one proxy by header hash, and an L7 proxy routing by URL path and retrying a failed backend, with the visibility each layer has annotated at each hop.
A single request crossing three distribution layers, DNS choosing a region, an L4 tier pinning a TCP connection to one proxy by header hash, and an L7 proxy routing by URL path and retrying a failed backend, with the visibility each layer has annotated at each hop.

Real-World Examples

  • Google published its Maglev design at NSDI 2016, describing the software network load balancer that has fronted Google services since 2008: a fleet of commodity machines each handling millions of packets per second at layer 4, using consistent hashing so that adding or removing a Maglev machine does not disturb the connection-to-backend mapping already in flight. It is a concrete illustration of the L4 tier existing specifically to absorb packet volume and distribute it to a smarter L7 tier behind it, rather than trying to do both jobs in one place.
  • AWS splits its managed load balancing into exactly this distinction and prices it accordingly: the Network Load Balancer operates at layer 4 and is documented as handling millions of requests per second with ultra-low latency and static IPs, while the Application Load Balancer operates at layer 7 and supports path-based and host-based routing rules, HTTP header conditions, and per-target-group health checks. Teams routinely deploy both, with an NLB fronting a fleet of self-managed L7 proxies such as Envoy or NGINX, for exactly the layered reason described above.
Key TakeawaysLoad balancing is not one component but a stack of decisions made at different layers, and each layer is defined by what it can see. DNS steers before any packet is sent but cannot react quickly. L4 moves packets at enormous rates but is blind to content. L7 can route, retry, and rate limit per request but pays CPU for every one. Design the request path by asking which layer is the cheapest place capable of making each decision you actually need.

Review Questions & Answers

The TTL is a hint, not an enforcement mechanism. Recursive resolvers, corporate DNS caches, operating system stub resolvers, and some application runtimes all cache records independently, and a meaningful number of them either impose a minimum TTL floor or cache indefinitely until restarted. That means clients can continue sending traffic to a removed address for minutes or hours after the DNS change. DNS is therefore appropriate for planned, gradual traffic shifts and for coarse geographic steering, while actual fast failover has to happen at L4 or L7 where the balancer can stop selecting a backend the moment a health check fails.

The L4 tier counts connections, not work. With HTTP/2 or gRPC, a single long-lived connection multiplexes many concurrent requests, so a backend holding the same number of connections as its peers can be handling many times more actual requests if those particular clients happen to be heavy. Long-lived connections also mean the distribution decision was made once, at connection establishment, and never revisited as the traffic mix changed. The fix is either to balance at L7, where the unit of load is the request, or to force periodic connection recycling so the L4 hash gets re-rolled.

Terminating TLS at the proxy means the proxy holds the private keys and sees every request in plaintext, which centralizes certificate management and enables content-based routing but also makes the proxy tier a high-value target and a place where a compromise exposes all traffic rather than one service. It also means the hop from proxy to application server is unencrypted unless you deliberately re-encrypt it, which is fine inside a trusted network segment and unacceptable across one you do not control. The tradeoff is real in both directions, which is why many organizations terminate at the edge and then re-establish mutual TLS internally.

Because the capabilities are not free. An L7 proxy must accept the connection, terminate TLS, parse the protocol, and hold per-request state, which costs orders of magnitude more CPU per packet than rewriting a header and forwarding. Putting a cheap L4 tier in front lets a small number of machines absorb enormous raw packet volume, including volumetric attack traffic that should never reach an expensive parser, and spread it across a horizontally scalable pool of L7 proxies. The L4 tier is also where techniques like direct server return and anycast VIPs live, neither of which an L7 proxy can offer.