How a system grows to meet load. Scale up vs. out, spread traffic with load balancing, and cut work with caching — the highest-leverage performance lever there is.
Why: there are two ways to handle more load — vertical scaling (a bigger machine: simple, but a hard ceiling and a single point of failure) and horizontal scaling (more machines: near-limitless and fault-tolerant, but requires the app to be stateless and distributable). When: scale up for a quick win, but architect for scaling out, which is the only path to large scale and high availability. Where: horizontal scaling requires designing statelessness in from the start.
VERTICAL (scale up) a bigger server
+ simple, no app changes
- hard ceiling, expensive at the top, single point of failure
HORIZONTAL (scale out) more servers behind a load balancer
+ near-limitless, fault-tolerant, commodity hardware
- requires STATELESS app instances (no local session/state)
Design for scale-OUT: keep instances stateless (state -> DB/cache/store).Why: horizontal scaling needs something to spread requests across the identical instances — a load balancer distributes traffic (round-robin, least-connections) and, crucially, health-checks instances so it stops routing to a dead one. When: put a load balancer in front of every horizontally-scaled tier. Where: it is also the seam for zero-downtime deploys and for removing failed nodes automatically, tying into availability.
┌──► instance 1
clients ──► [ LOAD BALANCER ] ──► instance 2 (identical, stateless)
└──► instance 3
- spreads traffic (round-robin / least-connections)
- HEALTH-CHECKS instances -> stops routing to a dead one (availability)
- enables rolling / zero-downtime deploys
Requires the instances to be interchangeable — hence "stateless".Why: the fastest, cheapest work is the work you avoid — caching stores the results of expensive operations (queries, computations, rendered pages) so repeat requests are served instantly, often improving performance and cost more than any other change. When: cache at every layer where data is read far more than written (browser, CDN, application, database). Where: the hard part is invalidation — a stale cache serves wrong data — so caching is a deliberate correctness/performance trade-off, not a free win.
Cache at every layer where reads >> writes:
browser ──► CDN (static/edge) ──► app cache (Redis) ──► DB ──► origin
closer to the user + fewer hops = faster + cheaper
The hard part is INVALIDATION — a stale cache serves WRONG data:
- TTL (expire after N seconds) - write-through / write-behind
- explicit invalidation on change - accept brief staleness where safe
Caching is the highest-leverage performance lever — and a correctness
trade-off. (See the Caching course for the patterns.)