Multimodal calls cost 10–100× a text call and take seconds. Deduplicate inputs, cache deterministic outputs, run heavy work async, and moderate what you generate.
Why: the same image or prompt often arrives repeatedly, and each call is expensive — content-addressing inputs by hash lets you skip work you already did. When: cache results whenever the same input deterministically yields the same output (a given image → its description). Where: the hash is both your cache key and your dedup key for stored files.
import hashlib
def content_key(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
cache: dict[str, str] = {}
def describe_image(data: bytes) -> str:
key = content_key(data)
if key in cache: # same bytes seen before -> no API call
return cache[key]
result = run_vision_call(data) # the expensive part
cache[key] = result
return resultWhy: transcribing an hour of audio or generating high-res images takes many seconds — doing it in a request handler blocks the user and risks timeouts. When: push long jobs to a background queue and return a job id the client polls or gets notified about. Where: this keeps your API responsive and lets you retry or rate-limit workers independently.
SYNCHRONOUS (bad for heavy work)
request --> [ 45s image/audio job ] --> response (client hangs, may time out)
ASYNC (right for heavy work)
request --> enqueue job --> return {job_id} (instant)
worker --> process --> store result
client --> poll /jobs/{id} OR receive a webhook
Meter usage per user so one caller can't run up the bill.Why: models can produce or transcribe content you must not show — generated images and transcripts are untrusted output that needs a safety pass before display. When: run moderation on generated/transcribed content before it reaches a user, and log what was blocked. Where: this is the same guardrail discipline as text, applied to pixels and audio.
from openai import OpenAI
client = OpenAI()
def safe_to_show(text: str) -> bool:
result = client.moderations.create(model="omni-moderation-latest", input=text)
return not result.results[0].flagged
# Gate transcripts and any text you display; use image moderation for pixels.
if not safe_to_show(transcript):
return {"error": "Content blocked by moderation."}