Cloud and DevOps
Load balancer
A load balancer is a server or service that spreads incoming traffic across a pool of backend servers, checks their health and routes around any that fail.
Teams put a load balancer in front of a service the moment one server is no longer enough, or the moment one server failing is no longer acceptable. It turns several machines into one address, and it is the first thing drawn on almost every architecture diagram.
When should you use a load balancer?
- More than one server for the same job. Clients need one address; the balancer hides how many servers are behind it.
- Availability targets. When one server fails, traffic must move to the others within seconds, without a human. See high availability.
- Zero-downtime deployments. Take one server out of the pool, update it, put it back, repeat.
- Scaling with demand. Add servers during a peak and remove them afterward.
What are the parts of a load balancer?
- Listener. The address, port and protocol the load balancer accepts traffic on, often with a TLS certificate.
- Pool or target group. The set of backend servers, containers or functions that can answer.
- Algorithm. The rule for choosing a backend for each connection or request.
- Health checks. Periodic probes that decide which pool members may receive traffic.
What load balancing algorithms are there?
| Algorithm | How it chooses | Use it when |
|---|---|---|
| Round robin | Each server in turn, with more turns for bigger servers when weighted | Servers and requests are uniform |
| Least connections | The server with the fewest open connections | Request durations vary |
| IP hash | The same client address always lands on the same server | Cheap persistence without cookies |
How do health checks work?
The load balancer probes every pool member on a schedule, such as GET /health every five seconds. A member that fails a set number of checks in a row is removed from rotation; one that passes a set number is added back. The health endpoint should check what the server needs to serve a request, such as its database connection, but not every downstream dependency, or one slow dependency takes the whole pool out. Deeper checks belong in the observability stack, not in the probe.
How do you read a load balancer diagram?
Follow the diagram at the top of this page from left to right. A browser, a mobile app and an API client all send HTTPS requests to one address, which is the load balancer. Its configuration is written inside the box: round robin, a health check on /health every five seconds and two failures before a server is marked unhealthy.
On the right, the first three requests go to servers 1, 2 and 3 in turn. Server 4 failed two consecutive checks, so it has been removed from rotation and gets no traffic. The dashed line along the bottom is the health check loop, which keeps probing server 4 and will add it back when it passes. Reading it this way raises the right questions: how many servers can fail before the rest are overloaded, and what does /health actually check.
What is the difference between layer 4 and layer 7 load balancing?
| Layer 4 | Layer 7 | |
|---|---|---|
| Works on | TCP and UDP connections | HTTP and HTTPS requests |
| Sees | Source and destination address and port | Path, host, headers, cookies, method |
| Can do | Forward a connection to a server | Route by content, terminate TLS, rewrite, add headers |
| Typical use | Databases, game servers, anything not HTTP | Web apps and APIs |
A layer 7 balancer is a reverse proxy with balancing built in, and an API gateway is a layer 7 balancer with API features added. Eraser’s engineering blog covers where each belongs in load balancing and API gateways.
What is session persistence?
Persistence, or sticky sessions, keeps a client on the same backend for the life of a session, usually with a cookie or by hashing the client address. It exists for servers that keep session state in memory, and it defeats the point of balancing: load becomes uneven and a server failure logs its users out. The better fix is to keep sessions in a shared store so any server can serve any request.
What is global versus regional load balancing?
A regional load balancer spreads traffic across servers in one region. A global load balancer sits in front of several regions and steers each client to one, by DNS or anycast, using latency, geography and the health of each region. The two are stacked. In a Kubernetes architecture or any container orchestration platform, a cloud load balancer in front of the cluster plays the regional role, a Service inside the cluster balances again across pods and a service mesh balances calls between the services of a microservices architecture.
What are common load balancer mistakes?
- Health checks that check too much. A probe that calls every dependency takes every server out when one dependency blinks.
- Health checks that check too little. Returning 200 from a process that cannot reach its database keeps a broken server in rotation.
- One load balancer. The component that exists to survive failures must survive its own.
What tools can you use to make a load balancer diagram?
- Cloud consoles and infrastructure code. Both describe listeners, pools and health checks precisely, as forms and resources rather than a picture.
- Drag-and-drop diagramming tools. Every provider icon set has a load balancer symbol, but each change to the pool is a manual edit.
- AI diagramming tools. Tools like Eraser turn infrastructure code or a description of your environment into an architecture diagram with load balancers, pools and health checks in the picture, which you can keep editing as servers come and go.
For more on this kind of diagram, see architecture diagrams.
Frequently asked questions
-
What is the difference between a load balancer and a reverse proxy?
Both sit in front of servers and forward requests. A reverse proxy is defined by what it does to each request: terminate TLS, cache, compress and route by path or host. A load balancer is defined by spreading traffic across a pool and routing around failures. Most products do both.
-
What is the difference between a load balancer and an API gateway?
A load balancer spreads identical requests across copies of the same service and knows little about them beyond address, port and perhaps path. An API gateway understands the request as an API call: it validates tokens, applies rate limits and rewrites paths. A gateway usually sits behind a load balancer and often has one in front of each service it routes to.
-
Which load balancing algorithm should I use?
Round robin is the default and is fine when every server is the same and every request costs about the same. Use least connections when request durations vary, so a server stuck on slow requests is not handed more. Use weighted variants when servers differ in capacity, and persistence only when a client must keep hitting the same server.
-
Is a load balancer a single point of failure?
It can be, which is why production load balancers run as a pair or a cluster with a shared virtual address and why cloud load balancers are built as distributed services across zones. The load balancer should be the most reliable component in the path, not the weakest.
