Catalog My learning Blog
System Design Roadmap: 100-Day Course for Engineers · Phase 6 · Lesson 54 · 4 min read

Service Discovery in Microservices

On this page
  1. Why hardcoded service addresses stop working
  2. What is a service registry?
  3. Client-side vs server-side service discovery
  4. Service discovery in Kubernetes
  5. Health checks and graceful shutdown
  6. Check yourself

Why hardcoded service addresses stop working

In a monolith on three fixed servers, the order service's config could say inventory = 10.0.1.15:8080. Now inventory runs in containers that autoscale between 4 and 20 instances, get rescheduled onto other machines, and receive new IP addresses on every deploy. A list in a config file is out of date within minutes. In microservices, services need a way to find each other at runtime.

What is a service registry?

A service registry is a live directory: for each service name, the addresses of instances that are currently running and healthy.

  • Instances register when they start (or the platform registers them).
  • The registry runs health checks, or instances send heartbeats; unhealthy or silent instances are removed.
  • Callers look up a name and get current addresses.

Examples: Consul, etcd, Netflix Eureka, and the built-in registries of Kubernetes and cloud platforms.

Client-side vs server-side service discovery

Client-side discovery, where the caller queries the registry, compared with server-side discovery, where a load balancer or proxy looks up instances for the caller.
Client-side discovery, where the caller queries the registry, compared with server-side discovery, where a load balancer or proxy looks up instances for the caller.

Client-side discovery. The caller queries the registry, gets the list of instances and picks one itself, usually with a client library that also load-balances and caches the list. No extra hop, and the client can make smart choices (least loaded, same zone). The cost: discovery logic lives in every client, in every language you use.

Server-side discovery. The caller sends requests to one stable name; a load balancer or proxy looks up healthy instances and forwards the request. Clients stay simple. The cost: an extra hop and another component to run.

Service meshes (Day 57) are a variant: a sidecar proxy next to each service does the lookups, so the application sees a stable name and the platform handles the rest.

Service discovery in Kubernetes

Kubernetes makes discovery mostly invisible:

YAML
apiVersion: v1
kind: Service
metadata:
  name: inventory
spec:
  selector:
    app: inventory          # every pod labeled app=inventory is a backend
  ports:
    - port: 80
      targetPort: 8080

The order service simply calls http://inventory/ (or inventory.shop.svc.cluster.local from another namespace). Cluster DNS resolves the name to a stable virtual IP, and Kubernetes routes each connection to one of the pods that's currently passing its readiness probe. Pods come and go; the name never changes. This is server-side discovery built into the platform.

One subtlety: with long-lived connections such as gRPC over HTTP/2, a connection sticks to one pod, so new pods may get no traffic. Use client-side load balancing over a "headless" service, or a mesh, for those.

Health checks and graceful shutdown

A registry is only as good as its health information. An instance that's registered but can't reach its database will receive traffic and fail it. Use readiness checks that reflect whether the instance can serve requests (Day 19), and remove instances on shutdown before they stop accepting connections, so in-flight requests finish cleanly.

Kubernetes has a known race here. When a pod is told to stop, two things happen at once: the container gets its stop signal, and the pod is removed from the Service's endpoints. Removal takes a few seconds to reach every node, so for that moment, new requests still arrive at a pod that's already shutting down, and fail. The common fix is a short preStop hook that sleeps a few seconds before shutdown begins, so routing catches up first:

YAML
lifecycle:
  preStop:
    exec: { command: ["sleep", "5"] }

ASP.NET Core then finishes in-flight requests during its own shutdown timeout.

Check yourself

Pods begin shutting down before every node stops routing to them. Add a short preStop sleep so endpoint removal propagates before the app stops accepting requests.

Client-side avoids an extra hop and allows smart balancing, but every client needs discovery logic. Server-side keeps clients simple at the cost of an extra component and hop.

It calls the Service name; cluster DNS resolves it, and Kubernetes routes each connection to a ready pod matching the Service's selector.

Instances come and go constantly with autoscaling, deploys and failures, and their addresses change. Hard-coded addresses point at instances that no longer exist. Service discovery lets callers find healthy instances by name at runtime.

A liveness probe asks whether the process is stuck and should be restarted. A readiness probe asks whether it can serve traffic right now, and failing it removes the pod from the service's endpoints without restarting it, for example while it warms up or its database is unavailable.