Design a Load Balancer

Beginner Infrastructure

The problem

Design a load balancer that sits in front of a fleet of web servers and distributes incoming requests across them.

You are in a 45-minute interview. Many candidates answer "use round robin". Interviewers want to see the whole picture.

The service is public and traffic varies through the day.

Think of the managed balancers in AWS (ALB and NLB) or software like Nginx and HAProxy.

Drive the conversation. Ask questions before drawing anything. State your assumptions out loud so they can be corrected early.
Use numbers. Turn the givens into requests per second, storage and bandwidth. Every later choice should point back to one of them.
Start simple, then evolve. Get a working version on the board first. Add caches, sharding and queues only when a number or a requirement asks for them.
Go deep on the hard parts. For this problem the hard parts are health checking, choosing an algorithm for uneven traffic and removing the balancer as a single point of failure.
Name the trade-offs. Say what you chose, what you gave up, and what would make you change your mind.
Think about failure. Point out single points of failure and say what happens when a node, a cache or a whole zone goes down.

Layer 4 vs Layer 7

L4 forwards TCP/UDP by IP and port (fast). L7 understands HTTP, so it can route by path, header or cookie, and terminate TLS.

Health check

A regular test (such as GET /health) used to decide whether a server should get traffic.

Connection draining

When removing a server, stop new requests but let in-flight ones finish.

Anycast

Many locations announce the same IP; the network sends each user to the nearest one.

Sticky session

Sending a user to the same server every time. Convenient, but it makes servers stateful.

Decisions you confirm will collect here as you go.

Your design so far