Do many things at once — threads for waiting, multiprocessing for heavy computation, and asyncio for high-volume I/O — and learn why the GIL shapes the choice.
Why: threads let your program do other work while it waits on slow things like downloads. ThreadPoolExecutor runs several tasks and collects their results without you managing threads by hand.
from concurrent.futures import ThreadPoolExecutor
import time
def fetch(url):
time.sleep(1) # pretend this is a slow download
return f'done: {url}'
urls = ['a', 'b', 'c']
with ThreadPoolExecutor() as pool:
results = list(pool.map(fetch, urls))
# all three "downloads" finish in ~1s, not 3s
print(results)Why: multiprocessing runs your code in separate processes, each with its own Python — sidestepping the GIL so heavy computation truly runs in parallel across CPU cores.
from concurrent.futures import ProcessPoolExecutor
def heavy(n):
return sum(i * i for i in range(n))
if __name__ == '__main__': # required on Windows
with ProcessPoolExecutor() as pool:
results = list(pool.map(heavy, [10_000_00, 10_000_00]))
print(results)Why: async lets a single thread juggle thousands of waiting tasks efficiently. Mark functions with "async def", pause on slow work with "await", and run many at once with asyncio.gather(). Great for web servers and API clients.
import asyncio
async def fetch(name):
await asyncio.sleep(1) # await = "pause here, let others run"
return f'done: {name}'
async def main():
results = await asyncio.gather(
fetch('a'), fetch('b'), fetch('c')
)
print(results) # all finish together in ~1s
asyncio.run(main())