Catalog My learning Blog
Scalability Patterns: Load Balancing, Caching, Sharding · Phase 1 · Lesson 5 · 10 min read

Global Load Balancing: Anycast and Geo-Routing

On this page
  1. Learning Objective
  2. Concept Explanation
  3. Worked Example: Routing Reads Near and Writes Home
  4. Real-World Examples
  5. Review Questions & Answers

Learning Objective

Compare the three mechanisms for steering traffic across regions, GeoDNS, anycast, and application-layer redirection, explain why routing a user to the geographically nearest region is not the same as routing them to the fastest or the correct one, and describe how data residency and stateful backends constrain global routing decisions.

Concept Explanation

Once a system serves users on more than one continent, the dominant latency term stops being anything your code does and becomes the speed of light. A round trip between London and Sydney has a hard floor near 250 milliseconds in fiber, and a page that makes even a handful of sequential requests across that distance feels broken regardless of how fast the servers are. Global load balancing exists to shorten that distance, and it has three main implementations, each with a different failure and control profile.

GeoDNS answers a DNS query differently depending on where the query appeared to come from, returning the address of the closest or otherwise preferred region. It is simple to operate, works with any protocol, and allows arbitrary policy, including weighted splits for gradual regional migration. Its two structural weaknesses are inherited from DNS itself. First, the authoritative server does not see the user, it sees the resolver, so a user whose device is configured to use a public resolver in a different country can be steered to the wrong region entirely. The EDNS Client Subnet extension mitigates this by passing a truncated client subnet along with the query, but support is not universal. Second, TTL caching means a regional failover propagates over minutes, which is far too slow to be the only failover mechanism.

Anycast takes the opposite approach and pushes the decision into the network itself. The same IP prefix is advertised via BGP from many locations, and the internet routing fabric delivers each packet to whichever advertisement is closest in BGP terms. Failover is fast and automatic, because withdrawing a route from a failed location causes traffic to reconverge on the remaining ones without any client involvement, and the client never learns that anything happened. The subtlety is that BGP closeness is an administrative and economic measure, not a geographic or latency one, so anycast routes a user to the topologically nearest location, which is usually but not always the fastest. Anycast is also naturally suited to stateless or short-lived interactions, which is why it is ubiquitous for DNS, CDNs, and DDoS absorption, and why TCP-terminating services using it rely on the fact that route changes mid-connection are rare rather than impossible.

Application-layer steering is the most precise and the most expensive. The client connects to some entry point, the service measures or looks up real conditions, and then redirects or instructs the client to use a specific regional endpoint. Because the decision is made with full information, it can incorporate measured latency from real user monitoring, current regional capacity and health, cost, and the one factor the other two mechanisms handle poorly: where this specific user data actually lives. Streaming and gaming platforms commonly use exactly this pattern, serving a lightweight manifest or matchmaking response that names the best server for this session.

The hardest constraints on global routing are not performance at all. Data residency regulation may legally require that a European user records be processed and stored in Europe, which overrides any latency-based preference and means the routing layer needs identity, not just geography, to decide. Statefulness imposes a similar constraint from a different direction: if a user data lives in one region primary database, routing their write to a nearer region either requires a cross-region write, which is slow, or multi-primary replication, which requires conflict resolution. The practical design pattern that reconciles these is to serve reads from a nearby replica while routing writes home to the region that owns that user data, which keeps the common case fast without pretending the consistency problem does not exist. It is also worth being explicit that a global architecture makes the system less available, not more, if a request must succeed in several regions to complete, since compounding dependencies multiply failure probability. Regions should be independent failure domains, not links in a chain.

MechanismWhere the Decision HappensFailover SpeedMain Weakness
GeoDNSAuthoritative nameserver, at resolution timeMinutes, bounded by TTL and resolver behaviorSees the resolver, not the user, unless client subnet is supported
AnycastThe BGP routing fabric, per packetSeconds, on route reconvergenceTopological distance is not the same as latency or geography
Application-layer steeringThe service itself, with full contextImmediate, on the next decisionCosts an extra round trip and requires bespoke client logic
Client-side selectionThe client, from a list plus its own measurementsImmediateTrusts the client and needs a distribution mechanism for the list

Worked Example: Routing Reads Near and Writes Home

The pattern below is what most globally deployed transactional systems converge on. It separates the read path, which can be served anywhere a replica exists, from the write path, which must reach the region that owns the record, and it makes the residency constraint an explicit input rather than an accident of geography.

Code
# Each user record carries its home region as data, not as an inference
# from the request source IP.
user.home_region  = "eu-west-1"     # set at signup, constrained by residency law
user.residency    = "EU"           # hard constraint: may not leave the EU

handle_request(req):
    edge = anycast_pop_that_received_this_packet()   # e.g. "ap-southeast-2"
    user = resolve_identity(req)

    if req.is_read():
        # Prefer a local replica, but only one inside the allowed boundary.
        candidates = replicas_for(user.home_region)
        legal      = [r for r in candidates if r.region in allowed(user.residency)]
        nearest    = min(legal, key=lambda r: measured_rtt(edge, r))
        if replication_lag(nearest) > user.max_tolerable_staleness:
            nearest = primary_for(user.home_region)   # fall back to the source
        return read_from(nearest)

    if req.is_write():
        # Writes always go home. No exceptions, no "nearby primary".
        return forward_to(primary_for(user.home_region), req)

# Latency budget for a Sydney user whose home region is eu-west-1:
#   read  from ap-southeast-2 replica ........   12 ms   (common case)
#   write forwarded to eu-west-1 primary .....  260 ms   (rare, and honest)
#
# The anti-pattern is making every request pay the 260 ms because the
# routing layer only knows "this packet arrived in Sydney" and nothing
# about which region actually owns this user's data.

# Availability math, the part teams get wrong:
#   Independent regions, request needs ANY one:   1 - (0.001)^3  = 99.9999%
#   Chained regions,   request needs ALL three:   (0.999)^3      = 99.7%
# Going global makes you LESS available if you build the second shape.
A global request path showing anycast delivering a packet to the nearest point of presence, a read served from a nearby compliant replica, and a write forwarded to the users home region primary, with the data residency boundary drawn as a hard constraint that overrides latency-based choice.
A global request path showing anycast delivering a packet to the nearest point of presence, a read served from a nearby compliant replica, and a write forwarded to the users home region primary, with the data residency boundary drawn as a hard constraint that overrides latency-based choice.

Real-World Examples

  • Cloudflare operates its entire network on anycast, advertising the same addresses from hundreds of cities so that a request is absorbed by the nearest point of presence and a datacenter going offline causes BGP to reconverge traffic onto neighbours without any client-visible failover step. The company has published detailed explanations of how this makes volumetric DDoS traffic spread across the whole network rather than concentrating on one target location, which is the availability argument for anycast rather than the latency one.
  • Netflix built its Open Connect content delivery network around placing appliances inside internet service provider networks and then steering each client to a specific appliance at play time, rather than relying on DNS alone, because the right server depends on which appliance actually holds the requested title, current appliance health, and measured throughput for that client. The published design is a clear instance of application-layer steering being chosen over pure geographic routing because the decision requires information only the application has.
Best PracticeNearest is not the same as best. GeoDNS sees a resolver rather than a user, anycast optimizes BGP topology rather than latency, and neither knows which region legally or architecturally owns a given user data. Make the home region an explicit property of the user record, serve reads from a nearby compliant replica, and route writes home, rather than letting the network layer make a data ownership decision it has no information to make.

Review Questions & Answers

The authoritative nameserver never sees the user, it sees whichever recursive resolver made the query. If the user device or network is configured to use a public resolver whose egress appears in North America, the GeoDNS policy is being applied to the resolver location, which is exactly what it was told. The mitigation is the EDNS Client Subnet extension, where the recursive resolver forwards a truncated portion of the client subnet along with the query so the authoritative server can make a decision based on the actual client network. Support is not universal, so the deeper mitigation is to not rely on DNS geography as the only steering mechanism: anycast makes the decision in the network where the real client path is visible, and application-layer steering can measure and correct after the fact.

With anycast, the destination of a packet is determined by the current BGP routing state, and that state can change mid-connection due to a route withdrawal, a peering change, or a reconvergence event elsewhere on the internet. If subsequent packets of an established TCP connection arrive at a different location than the one holding the connection state, the connection breaks. For DNS, where an interaction is usually a single small UDP exchange, this is a non-issue. For short HTTP requests it is rare enough to be handled by a retry. For a long-lived connection such as a large upload, a streaming session, or a websocket, the exposure window is much longer, which is why such systems either accept and handle the reset, use anycast only to reach an entry point and then redirect to a unicast address, or rely on protocols like QUIC that carry a connection identifier allowing migration across paths.

Availability multiplies when dependencies are chained. If completing a request requires both region A and region B to be working, the combined availability is the product of the two, which is strictly lower than either one alone. Teams reach this shape accidentally: a service deployed in a new region still calls a database, an auth service, or a feature flag service that only exists in the original region, so every request now depends on two regions instead of one. The property that prevents it is region independence, meaning each region contains everything needed to serve a request end to end, with no synchronous cross-region dependency on the critical path. Then availability is one minus the product of the failure probabilities, which improves with each region added rather than degrading. Cross-region traffic should be asynchronous replication, not a synchronous call.

Geographic routing decides based on where the request came from, but the regulation constrains where a specific user data may live, and those are different facts. A European user travelling in Singapore still has data that must remain in the EU, and a Singapore user connecting through a European VPN egress must not have their data pinned to the EU because a router thought they were there. Request origin is a fluctuating property of a network path; residency is a durable property of the account. The system needs residency and home region stored as attributes of the user record, established at account creation and enforced at the data layer, with the routing tier consulting identity rather than inferring from source IP. Geography then becomes an optimization applied only within the set of endpoints the residency constraint already permits.