A prompt is production logic — treat it like code, not a hardcoded string. Version prompts, record which version served each request, and roll changes out safely.
Why: a prompt buried inline as a string is untrackable — you cannot tell which version caused a regression or roll back a bad change. When: store prompts in source control (or a prompt registry) with a version id, exactly like any other deployable code. Where: this is what lets you tie a quality drop in your traces to a specific prompt change.
# prompts.py — versioned, reviewable, diff-able in source control.
PROMPTS = {
"support_agent": {
"version": "2025-07-06.3",
"system": (
"You are a support agent. Answer only from the provided context. "
"If the answer isn't there, say you don't know. Be concise."
),
},
}
def get_prompt(name: str) -> dict:
return PROMPTS[name]Why: recording the prompt version in each trace is what makes a regression debuggable — you can see the exact wording that produced any output. When: log the version alongside model, tokens, and latency on every call. Where: without this, "quality dropped last Tuesday" is unsolvable; with it, you diff two prompt versions.
def answer(question: str, context: str):
p = get_prompt("support_agent")
r = client.messages.create(
model="claude-opus-4-8", max_tokens=512, system=p["system"],
messages=[{"role": "user", "content": f"{context}\n\n{question}"}],
)
log_trace(prompt_version=p["version"], model="claude-opus-4-8",
tokens=r.usage.output_tokens) # version in every trace
return r.content[0].textWhy: a new prompt can regress in ways your eval set missed, so ship it to a slice of traffic first and keep a one-step rollback. When: gate the rollout on evals passing, canary to a small percentage, watch the traces, then ramp — or revert instantly if quality drops. Where: because the previous version is still in source control, rollback is changing one id, not an emergency.
Safe prompt release:
1. Change prompt -> evals must pass in CI (gate the deploy)
2. Canary: route 5% of traffic to the new version
3. Watch traces: quality, cost, latency vs. the old version
4. Ramp to 100% if healthy
5. Regression? Flip the active version id back -> instant rollback