When a big system prompt or document is reused across calls, caching it cuts cost and latency dramatically. Mark the reusable prefix and understand what actually gets cached.
Why: if every request resends a 5,000-token system prompt or reference document, you pay to reprocess it every time — caching stores that prefix so repeat calls read it back at a fraction of the price and latency. When: cache any large, stable prefix reused across requests (system prompts, tool definitions, a long document, few-shot examples). Where: only the unchanging prefix is cached; the varying user turn is always fresh.
Request shape that caches well:
[ big STABLE prefix — system prompt, docs, examples ] <- cache this
[ small CHANGING suffix — this user's question ] <- always fresh
Cache writes cost a little more; cache reads are much cheaper
(and faster). Reuse the prefix enough and it pays for itself fast.Why: you tell the API where the reusable boundary is by adding cache_control to the last block you want cached — everything up to that point becomes the cached prefix. When: put the breakpoint at the end of your stable content, before anything that changes per request. Where: the first call writes the cache; subsequent calls within the TTL read it.
LONG_DOC = open("handbook.txt").read() # imagine ~10k tokens, reused all day
response = client.messages.create(
model="claude-opus-4-8", max_tokens=1024,
system=[
{"type": "text", "text": "You answer questions about the handbook."},
{
"type": "text",
"text": LONG_DOC,
"cache_control": {"type": "ephemeral"}, # cache up to here
},
],
messages=[{"role": "user", "content": "What's the vacation policy?"}],
)Why: caching silently does nothing if the prefix changes even slightly between calls — the usage fields tell you whether you wrote or read the cache. When: check cache_read_input_tokens on the second identical call; if it is zero, your prefix is not byte-identical. Where: a moving timestamp or reordered tool list is the usual reason a cache never hits.
u = response.usage
print("cache write:", u.cache_creation_input_tokens) # >0 on the first call
print("cache read: ", u.cache_read_input_tokens) # >0 on later calls
print("uncached in:", u.input_tokens) # just the changing suffix
# Rule of thumb: keep the cached prefix byte-for-byte identical across calls.
# Any change before the cache_control breakpoint invalidates the cache.