Catalog My learning Blog
Computer Networking for Developers: TCP, DNS, HTTP, TLS · Phase 4 · Lesson 25 · 3 min read

Service Discovery in Distributed Systems

On this page
  1. Why service discovery exists
  2. The service registry
  3. Client-side vs server-side discovery
  4. Kubernetes: DNS plus a virtual IP
  5. Service discovery pitfalls
  6. Check yourself

Why service discovery exists

With three servers that never change, you can put their IPs in a config file. With autoscaling, deploys, containers and failed instances replaced automatically, addresses change many times a day. Service discovery answers one question at runtime: where are the healthy instances of the orders service right now?

The service registry

Discovery systems keep a service registry: a live list of instances and their addresses.

  • Registration: instances add themselves when they start (self-registration), or the platform registers them (Kubernetes, ECS, Nomad).
  • Health: instances send heartbeats or the registry runs health checks; failing instances are removed (Day 20 of the distributed systems course covers the detection trade-offs).
  • Deregistration: instances remove themselves on clean shutdown, and expire on crashes.

Consul, etcd, ZooKeeper and Eureka are common registries; Kubernetes uses its own API server backed by etcd.

Instances register with a registry and send heartbeats; callers ask the registry for healthy addresses, then call an instance directly.
Instances register with a registry and send heartbeats; callers ask the registry for healthy addresses, then call an instance directly.

Client-side vs server-side discovery

Client-side discoveryServer-side discovery
HowThe caller queries the registry and picks an instance itselfThe caller sends to a stable address; a load balancer or proxy picks the instance
Load balancingIn the client libraryIn the load balancer
ProsNo extra hop; smart client-side balancingClients stay simple and language-agnostic
ConsEvery language needs the libraryAn extra component to run and scale
ExamplesNetflix Eureka with Ribbon, gRPC client-side balancingKubernetes Services, AWS load balancers, service meshes

Kubernetes: DNS plus a virtual IP

Every Kubernetes Service gets a stable DNS name and a virtual IP. From any pod:

Shell
$ nslookup orders.default.svc.cluster.local
Name:    orders.default.svc.cluster.local
Address: 10.96.24.17

The name follows <service>.<namespace>.svc.cluster.local, and within the same namespace just orders works. 10.96.24.17 is a ClusterIP: no pod has it. kube-proxy (or an eBPF data plane like Cilium) programs every node to send traffic for that IP to one of the ready pods behind the Service. Pods are listed in EndpointSlices, and only pods passing their readiness probe receive traffic.

A headless Service (clusterIP: None) skips the virtual IP: DNS returns the pod IPs directly, which is what databases and gRPC clients doing their own balancing need.

Service discovery pitfalls

  • DNS caching: clients that cache DNS answers too long, or connection pools that never reconnect, keep calling instances that are gone. Keep TTLs short and recycle connections.
  • Long-lived connections defeat balancing: HTTP/2 and gRPC reuse one connection, so a ClusterIP balances per connection, not per request. All traffic can pile onto one pod. Use client-side balancing over a headless Service, or a mesh that balances per request.
  • Health checks that check too much remove every instance when one shared dependency is slow.
  • A service mesh (Istio, Linkerd) adds a proxy next to each instance that handles discovery, per-request balancing, retries and mutual TLS, at the cost of extra components.

Check yourself

In client-side discovery the caller queries the registry and picks an instance; in server-side discovery it calls a stable address and a load balancer picks.

A virtual IP for a Service that no pod owns; each node routes traffic for it to one of the Service's ready pods.

gRPC keeps one long-lived HTTP/2 connection and sends all requests over it, while a ClusterIP balances connections, not requests, so that one connection pins all traffic to one pod. Use client-side load balancing with a headless Service, or a service mesh.

A Service with no ClusterIP. DNS returns the pods' own IPs, so clients can connect to them directly and balance requests themselves.

A registry listing dead instances keeps sending traffic into failures. Health checks or heartbeats with short expiry remove unhealthy instances quickly, so clients only discover instances that can serve.