Load Balancing Algorithms
The Question
What are the common load balancing algorithms?
What a Strong Answer Covers
- At least four named. Least connections use case.
Senior-Level Answer
Load balancing algorithms determine how incoming requests are distributed across a pool of backend servers. The right choice depends on request duration variance, session affinity requirements, and backend heterogeneity.
**Round Robin** distributes requests sequentially across all backends. It is simple, stateless, and works well when requests have similar processing times and servers have equivalent capacity. Weighted Round Robin extends this by assigning a weight to each backend, directing proportionally more traffic to higher-capacity servers.
**Least Connections** routes each new request to the server with the fewest active connections. This adapts well to variable request duration — if some requests take much longer than others, round robin would overload the servers handling those slow requests. Least Outstanding Requests is a variant that tracks in-flight requests rather than established connections, useful for HTTP/2 multiplexed streams.
**IP Hash (Sticky)** hashes the client IP address to consistently route a given client to the same backend. This provides session affinity without application-level session sharing, but it can cause uneven distribution when a small number of IPs generate disproportionate traffic, and it breaks if a backend is added or removed (use consistent hashing to mitigate this).
**Consistent Hashing** maps both requests (by a key such as URL, user ID, or IP) and servers onto a hash ring. When a server is added or removed, only requests that were mapped to that server are remapped — minimizing cache invalidation in systems like distributed caches or CDN origin selection.
**Random** selects a backend uniformly at random. With Power of Two Choices (P2C), two backends are sampled randomly and the one with fewer active connections is chosen — this achieves near-optimal balance with O(1) overhead, used in Envoy and Nginx.
**Resource-Based / Adaptive** algorithms use real-time metrics (CPU, memory, response latency) reported by agents on each backend. This is the most accurate but most complex, requiring a health reporting infrastructure.
In practice, least connections or P2C is the default choice for most microservice deployments. Round robin is sufficient for stateless, uniform request loads. Consistent hashing is essential when backend affinity must survive partial pool changes.
What Separates a 2/3 from a 3/3
2/3 — Passing but Incomplete
Names and correctly describes round robin, least connections, and IP hash. May not distinguish weighted variants or consistent hashing.
3/3 — Strong Answer
Covers at least five algorithms with clear use case distinctions, explains when each fails (e.g., IP hash with skewed traffic), mentions consistent hashing and why it is preferred over plain IP hash for cache affinity, and references real implementations (Envoy, Nginx).
Common Mistakes
- Listing algorithms without explaining which traffic pattern each is optimized for.
- Confusing IP hash with consistent hashing — IP hash is a simple modulo operation that breaks badly on pool changes.
- Forgetting that round robin fails when request durations vary significantly.
- Not mentioning the P2C (Power of Two Choices) optimization for random selection.
Follow-Up Questions
- How does consistent hashing minimize cache misses when a backend is added or removed? — Only the keys mapped to the removed/added node are remapped — O(K/N) keys instead of O(K) with modulo hashing.
- What is session affinity and what are its failure modes in a load-balanced system? — Affinity routes a client to the same backend, required for in-process session state. Failure modes: backend restart loses session, uneven load if client distribution is skewed.
- How would you implement health checks in a load balancer and what happens when a backend fails mid-request? — Active checks (HTTP probes) and passive checks (error rate). In-flight requests to a failing backend are typically aborted; the load balancer should retry on a healthy backend if the request is idempotent.
- How does L4 load balancing differ from L7 load balancing? — L4 operates on TCP/UDP — routes based on IP/port without inspecting payload. L7 can route based on HTTP headers, URL path, cookies — enabling more sophisticated strategies but at higher CPU cost.
Related Questions
Can You Explain This Cold?
Reading the answer is step one. Explaining it unprompted — under interview pressure — is what actually matters. Get AI-graded feedback on your answer with follow-up probes on your weak points.
Get Graded — Free Assessment