Load Balancing Algorithms In Depth

Nginx In Depth

Chapter 5 · Load Balancing Algorithms In Depth

Web Servers Fundamentals Chapter 6 introduced only plain round robin, and Chapter 4 of this course added weighting on top of it. Nginx supports two genuinely different algorithms beyond that — each solving a different real problem round robin doesn't handle well.

Round Robin, Revisited

Round robin (Nginx's own default, with no algorithm directive needed) cycles through the servers in an upstream block in order. With weight (Chapter 4), it becomes weighted round robin — the cycle still goes in order, but a higher-weighted server appears more often within that cycle. Round robin distributes request count evenly (adjusted by weight); it has no idea whether any individual request is fast or slow.

least_conn

upstream backend_pool {
    least_conn;
    server 10.0.0.11:3000;
    server 10.0.0.12:3000;
}

least_conn routes each new request to whichever backend currently has the fewest active connections, rather than blindly cycling through the list. This matters when requests take meaningfully different amounts of time to process: round robin can pile several long-running requests onto one server purely by coincidence of ordering, leaving another server comparatively idle, while least_conn actively avoids sending new work to an already-busy server.

ip_hash

upstream backend_pool {
    ip_hash;
    server 10.0.0.11:3000;
    server 10.0.0.12:3000;
}

ip_hash routes based on a hash of the client's own IP address, so the same client consistently lands on the same backend across requests — solving session persistence ("sticky sessions") for an application that stores session state locally in each backend's own memory rather than in a shared store. The trade-off: if many clients share one IP address (common behind a large corporate NAT or proxy), those clients all get hashed to the same backend, producing an uneven distribution that has nothing to do with actual load.

The General hash Directive

upstream backend_pool {
    hash $cookie_session_id consistent;
    server 10.0.0.11:3000;
    server 10.0.0.12:3000;
}

hash generalizes ip_hash's own idea to any key, not just client IP — a session cookie value, as shown above, or a URL, letting the same key consistently map to the same backend regardless of which client IP it came from. The optional consistent parameter uses a consistent-hashing algorithm, meaning adding or removing a backend server reshuffles only a small fraction of existing key-to-backend mappings, rather than potentially reshuffling nearly all of them.

Algorithm	Directive	Best fit
Round robin (weighted)	default + `weight`	Uniform, short-lived requests
Least connections	`least_conn`	Requests with variable processing time
IP hash	`ip_hash`	Simple sticky sessions by client IP
General hash	`hash`	Sticky sessions or cache-locality by any key

A real scenario where this stops being optional

A Node.js application storing session data purely in each server process's own memory (no shared session store) genuinely needs ip_hash or hash to function correctly under load balancing at all — without it, a user's session data could exist only on the one backend instance that happened to handle their first request, with every subsequent request potentially landing on a different instance that has never heard of that session.

Sticky sessions are a workaround, not a substitute for shared session storage

It's tempting to treat ip_hash/hash-based sticky sessions as just as good as storing session state in a shared store like Redis. It's a real, useful workaround, but it has genuine limitations this chapter's own material already exposed: uneven distribution behind a large NAT, and — more seriously — if the one backend holding a client's session data goes down, that session is lost entirely, with no other backend able to pick it up. Externalizing session state into a shared store instead makes every backend genuinely stateless, so any load-balancing algorithm — including plain round robin — works safely, and a backend failing doesn't destroy anyone's session.

Hands-On Exercises

Exercise 1

A backend pool handles requests that sometimes take 50ms and sometimes take 8 seconds, with no predictable pattern to which requests are slow. Which load balancing algorithm from this chapter would you recommend, and why would plain round robin be a worse fit here?

📄 View solution

Exercise 2

A company using ip_hash notices that a disproportionate share of their traffic always lands on the same one or two backend servers, even though total request volume is evenly spread across many different users. Using this chapter's own material, explain the likely cause.

📄 View solution

Exercise 3

A team relies on ip_hash for session persistence and considers this "just as reliable" as using a shared session store like Redis. Using this chapter's own warning box, explain what real risk this team is exposed to that a shared session store would eliminate.

📄 View solution

Chapter 5 Quick Reference

Round robin (default, + weight) — distributes request count evenly, blind to how long each request takes
least_conn — routes to whichever backend has the fewest active connections; better for variable-duration requests
ip_hash — sticky sessions by client IP; risks uneven distribution behind a shared-IP NAT
hash (optionally consistent) — sticky sessions or cache-locality on any key, not just IP
Sticky sessions are a workaround — a shared session store makes backends genuinely stateless and safe under any algorithm