Catalog My learning Blog
System Design Roadmap: 100-Day Course for Engineers · Phase 6 · Lesson 57 · 4 min read

Service Mesh Explained: Istio and Linkerd

On this page
  1. Forty services, forty retry implementations
  2. How a service mesh works: data plane and control plane
  3. What a service mesh gives you
  4. Worked example: a canary release with Istio
  5. Service mesh drawbacks
  6. When a service mesh is worth it
  7. Check yourself

Forty services, forty retry implementations

By now every service needs the same networking features: discovery, timeouts, retries, circuit breaking, encryption between services, and metrics for every call. A platform with 40 services in three languages ends up with 40 slightly different implementations of all of it, maintained by 40 teams. A service mesh moves that logic out of application code into the infrastructure.

How a service mesh works: data plane and control plane

Data plane: a small proxy, called a sidecar, runs next to every service instance, and every request in and out of the service passes through it. Envoy is the most common proxy; Linkerd uses its own, written in Rust. The application thinks it's calling http://inventory; the sidecar does the discovery, encryption, retries and measurement.

Control plane: a central component that pushes configuration to all the sidecars: routing rules, retry policies, security policies, certificates.

Two services, each with a sidecar proxy handling routing, retries and encryption, configured centrally by a control plane.
Two services, each with a sidecar proxy handling routing, retries and encryption, configured centrally by a control plane.

Newer designs reduce per-pod overhead: Istio's ambient mode replaces per-pod sidecars with a shared per-node proxy for basic features, adding heavier proxies only where needed.

What a service mesh gives you

  • Mutual TLS everywhere: traffic between services is encrypted, and each service gets a cryptographic identity, without application changes. Policies like "only checkout may call payments" become enforceable (Day 84).
  • Uniform resilience: timeouts, retries and circuit breaking configured as policy, consistently.
  • Traffic control: send 5 percent of traffic to a new version (canary), mirror traffic to a test version, route by header.
  • Telemetry for free: request rates, error rates and latency for every service-to-service call, plus tracing headers propagated.

Worked example: a canary release with Istio

Version 2 of the inventory service is deployed next to version 1. With Istio, a routing rule sends a small share of traffic to it:

YAML
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: inventory
spec:
  hosts: ["inventory"]
  http:
    - route:
        - destination: { host: inventory, subset: v1 }
          weight: 95
        - destination: { host: inventory, subset: v2 }
          weight: 5
      timeout: 2s
      retries: { attempts: 2, perTryTimeout: 500ms }

Watch v2's error rate and latency in the mesh's metrics. If they're healthy, move to 25, 50, then 100 percent. If not, set v2's weight to 0. No application code or client changes are involved.

Service mesh drawbacks

  • Another complex system to install, upgrade, secure and debug, with its own failure modes.
  • Latency and resources: an extra proxy hop on each side of every call, usually around a millisecond, plus memory and CPU per proxy.
  • Harder debugging: a failure might now be in the app, the sidecar or the mesh configuration.
  • Skill: someone must understand it deeply.
  • Double retries. If the application already retries with Polly and the mesh also retries, one failure becomes attempts multiplied by attempts. Choose one layer to own retries and timeouts, usually the mesh once you adopt it, and remove the other.
  • Sidecar lifecycle. The app can start before its sidecar is ready and fail its first calls, and a batch job can finish while the sidecar keeps the pod running. Recent Kubernetes versions support native sidecar containers that start first and stop last; on older clusters, you need workarounds.

When a service mesh is worth it

A mesh pays off with many services, several languages, and strong requirements for encryption between services or fine-grained traffic control. With five services in one language, a good shared library and a gateway are usually simpler. Linkerd aims for simplicity and low overhead; Istio offers more features at more complexity. Managed meshes from cloud providers reduce the operational load.

Check yourself

Both layers retry, so each request becomes 3 app attempts times 3 mesh attempts. Let one layer own retries and turn them off in the other.

Mutual TLS between services, so traffic is encrypted and identities are verified; uniform retries, timeouts and circuit breaking configured centrally; and traffic splitting for canary releases. It also produces consistent per-call metrics and tracing data for every service.

With a small number of services in one language, where a shared library and gateway cover the needs without the mesh's operational cost and complexity.

The data plane is the set of proxies, usually sidecars, that sit next to each service and handle every request. The control plane configures those proxies: routing rules, certificates and policies. Applications talk only to their local proxy.

Extra latency and resources from proxies on every hop, a complex system to run and upgrade, and a new place for outages. For a handful of services, libraries and a gateway are often simpler.