Service mesh

Teams add a service mesh when the same networking concerns keep reappearing in every service: encrypt this call, retry that one, time out the slow one, trace all of them. Moving that work into proxies gets it done once, consistently, without a library upgrade in forty repositories.

What problem does a service mesh solve?

In a microservices architecture, most of the risk lives on the network between services. Each team has to decide how to secure calls, what to do when a dependency is slow and how to see a request that crosses six services. Done in code, those decisions are made differently by every team and language. A mesh makes them platform decisions, applied by the proxies to every call and changed with configuration rather than deploys.

What are the parts of a service mesh?

• Data plane. The proxies that carry the traffic. In the sidecar model there is one beside every service instance, and the service only ever talks to its own proxy.
• Control plane. The components that configure the proxies: a policy and routing store, a certificate authority that issues service identities and service discovery. It never touches request traffic.
• Telemetry pipeline. Each proxy emits metrics, access logs and trace spans for every call, which flow to the observability stack.
• Ingress and egress gateways. Reverse proxies at the edge of the mesh that bring outside traffic in and send calls out, often alongside or merged with the API gateway.

What does a service mesh give you?

Capability	What the proxies do	What it replaces
Mutual TLS	Encrypt and authenticate every call with workload certificates	Per-service TLS code and certificate handling
Traffic management	Retries, timeouts, circuit breaking, canary and weighted routing	Resilience libraries in every language
Authorization	Allow or deny calls by source identity and path	Network rules and ad hoc checks
Observability	Uniform metrics, logs and distributed traces for every hop	Instrumentation written per service

None of this requires a code change in the services, and it applies equally to the legacy service nobody wants to touch.

How does a service mesh work?

Follow a call in the diagram at the top of this page. The order service calls the payment service, but the call to localhost is answered by its sidecar proxy. The sidecar looks up where the payment service’s instances are, opens a mutual TLS connection to the payment sidecar using the certificate the control plane issued, applies the configured timeout and retry policy and forwards the request. The payment sidecar checks that orders is allowed to call payments, then hands the request to the payment service on localhost.

Both sidecars record the call: latency, status, source and destination identity and a trace span. Those flow to the observability system, so a trace of one checkout shows the hop from orders to payments to inventory with timings for each. The control plane sits above all of this, pushing routing rules and fresh certificates to every sidecar but never sitting in the request path.

Sidecar or sidecarless?

The sidecar model puts a proxy in every pod. It gives the finest control and the strongest isolation, but every pod pays for a proxy’s memory and an extra hop, and upgrading the mesh means restarting every pod. Sidecarless or ambient meshes run one proxy per node for layer 4 work such as mTLS and add a layer 7 proxy only where a service’s rules need to read the request, which cuts the overhead and the upgrade pain. Both run on the same Kubernetes architecture or other container orchestration platform; the choice is about cost and control, not capability.

What does a service mesh cost?

• Resources. A proxy per pod can add tens of megabytes of memory and a few milliseconds of latency per hop, multiplied by the pods and hops in a request.
• Operations. The control plane is another critical system to upgrade, monitor and keep highly available. See high availability.
• Debugging. A failure may now be in the service, the sidecar, the policy or the certificate. Good tracing is the way out.

How do you adopt a service mesh?

1. Start with observability. Install the mesh in permissive mode and use the uniform metrics and traces before changing any traffic behavior.
2. Turn on mTLS gradually. Allow plain text while sidecars roll out, then require mTLS namespace by namespace.
3. Add timeouts and retries per route. Keep retries off for calls that are not idempotent.
4. Write authorization policies. Start from deny by default for the most sensitive services, such as payments.
5. Diagram it. Put the mesh on the architecture diagram: which services are in it, where the gateways are and what the policies between them say.

What are common service mesh mistakes?

• Adopting it too early. With five services, the mesh costs more than it saves.
• Retries on non-idempotent calls. A retried payment is a double charge.
• Treating the mesh as a load balancer replacement. The mesh balances calls between services; traffic from the internet still needs a load balancer or gateway in front.

What tools help with a service mesh?

• The mesh’s own tooling. Istio, Linkerd and the cloud providers’ managed meshes ship dashboards and a CLI that show the services they know about and the policies between them.
• Drag-and-drop diagramming tools. Good for an explanatory picture of the sidecar pattern; less good for keeping up with a live mesh.
• AI diagramming tools. Tools like Eraser turn a description of your services and the policies between them into an architecture diagram with the mesh, the gateway and the observability stack in the picture, which you can keep editing as services are added.

For more on this kind of diagram, see Kubernetes architecture diagrams.

Frequently asked questions