The thread-per-connection model hits a wall at scale — the C10k problem. Understand blocking I/O, why it wastes threads waiting, and how non-blocking I/O lets one thread serve thousands of connections.
By default, a socket read blocks: the thread stops and waits until data arrives. In a thread-per-connection server, most threads spend most of their time blocked, doing nothing but consuming memory (each thread’s stack) and scheduling overhead. With thousands of mostly-idle connections — common for a game — this model wastes enormous resources on threads that are just waiting.
Blocking I/O: recv() STOPS the thread until data arrives.
thread-per-connection server with 10,000 mostly-idle players:
-> 10,000 threads, each ~1 MB of stack = ~10 GB just for stacks
-> the OS scheduler thrashes context-switching between them
-> nearly all of them are BLOCKED (waiting), doing no work
Most connections are idle most of the time (waiting for the next packet). Paying
for a whole thread PER idle connection is enormously wasteful. This is the wall
the thread-per-connection model hits.The historical name for this wall is the C10k problem: how to handle ten thousand concurrent connections on one server. The answer was to stop using a thread per connection and instead use non-blocking I/O with a single thread (or a few) that juggles all connections. Modern servers handle far more than 10k this way. This shift from "thread per connection" to "event loop" is the foundation of scalable network servers.
The C10k problem (c. 1999): can one server handle 10,000 concurrent connections?
thread-per-connection -> NO (memory + context-switch cost explodes)
the answer: DON'T use a thread per connection. Use a FEW threads with
NON-BLOCKING I/O, driven by an EVENT LOOP that watches all connections at
once and only does work when a connection is actually ready.
-> one thread can juggle tens of thousands of connections, because it only
touches the ones with data waiting (no idle threads).
Every high-scale server (nginx, Node.js, Redis, game servers) is built this way.
Next: how the event loop knows which connections are ready.The building block is a non-blocking socket: instead of waiting, a read returns immediately — with data if some is ready, or a "would block" signal if not. This lets one thread poll many sockets without getting stuck on any single one. On its own, spinning over thousands of sockets checking each is wasteful, which is why non-blocking sockets are paired with an efficient readiness notification mechanism (the next lesson).
import socket
sock.setblocking(False) # non-blocking: calls return immediately
try:
data = sock.recv(1024) # returns data if ready...
except BlockingIOError:
pass # ...or raises immediately if nothing's ready
# (instead of BLOCKING the thread)
# now one thread can check MANY sockets without getting stuck on any one.
# but naively looping over 10,000 sockets asking "ready? ready?" burns CPU.
# we need the OS to TELL us which sockets are ready -> I/O multiplexing (next).Async, event-loop I/O scales to huge connection counts with tiny memory, but it changes how you write code: you cannot just call a blocking function, and CPU-heavy work in the loop stalls every connection. Threads are simpler to reason about and use multiple cores for CPU work but do not scale to massive connection counts. Real servers often combine them: an event loop for the many network connections, plus a thread/process pool for heavy computation.
Threads vs event loop — the trade-off:
THREADS (blocking) simple linear code, use multiple CPU cores for work,
but ~1 thread per connection -> limited scale.
EVENT LOOP (non-blocking) one thread, tens of thousands of connections, tiny
memory — BUT no blocking calls allowed, and CPU-heavy
work in the loop stalls EVERY connection.
Real servers COMBINE them:
event loop -> handle the many network connections (I/O-bound)
thread/process pool -> offload heavy CPU work (physics, pathfinding) so it
doesn't block the loop.
Match the tool to the work: async for I/O concurrency, threads/processes for
CPU parallelism.