Multimodal inputs are big, binary, and user-supplied — validate before you spend a single token. Check type and size, encode correctly, and set the right timeouts.
Why: multimodal calls are expensive and slow, and users upload the wrong thing constantly — checking type, size, and format up front rejects bad input for free instead of after a costly failed call. When: validate at the boundary, before any encoding or API call. Where: never trust a file extension alone; check the actual bytes.
from pathlib import Path
ALLOWED = {"image/jpeg", "image/png", "image/webp"}
MAX_BYTES = 10 * 1024 * 1024 # 10 MB
def validate_image(path: str):
p = Path(path)
if p.stat().st_size > MAX_BYTES:
raise ValueError("File too large — resize before uploading.")
# Sniff the real type from magic bytes (e.g. python-magic), not the extension.
import magic # pip install python-magic
mime = magic.from_file(path, mime=True)
if mime not in ALLOWED:
raise ValueError(f"Unsupported type: {mime}")
return mimeWhy: how you attach a file depends on the API — inline base64 for small local files, a URL for remote assets, multipart upload for audio endpoints — and getting it wrong is a common source of 400s. When: pick inline for small one-off images, URLs when the asset is already hosted. Where: base64 inflates size ~33%, so large files favor URLs or upload endpoints.
SMALL local image -> base64 inline in the message (simplest)
REMOTE asset -> pass a URL source (no re-upload)
AUDIO file -> multipart file upload (transcription/speech APIs)
base64 adds ~33% size. For big files prefer URLs or upload endpoints.Why: multimodal calls take much longer than text, so the default text timeout will abort valid requests — raise it, and normalize the response before the rest of your app touches it. When: bump client timeouts for image and audio calls specifically. Where: handle content-policy refusals and unsupported-format errors as structured results, not crashes.
from openai import OpenAI
# Longer timeout than text calls — image/audio generation is slow.
client = OpenAI(timeout=120.0)
try:
result = client.images.generate(model="gpt-image-1",
prompt="a mint leaf logo", size="1024x1024")
except Exception as e:
# Content policy, unsupported format, timeout -> handle, don't crash.
return {"error": str(e)}