Hosted or open source, big or small, general or domain-specific — the embedding model sets your retrieval ceiling. Compare the options and pin the one decision that must never drift.
Why: retrieval quality is capped by your embedding model, so the choice is not cosmetic — but bigger is not automatically better for your domain. When: weigh dimension (storage + speed), quality on YOUR data, cost, and whether it can run locally. Where: the MTEB leaderboard ranks models, but always test on your own queries before trusting it.
DIM HOSTED? NOTES
all-MiniLM-L6-v2 384 local tiny, fast, great baseline
bge-large-en 1024 local stronger, heavier
OpenAI text-3-* 1536+ hosted high quality, per-token cost
Cohere embed-v3 1024 hosted strong multilingual
Voyage / Gemini varies hosted domain + multilingual options
Higher dim = more storage and slower search, not always better recall.Why: a hosted API gives you top-tier quality with no model to run, at a per-token cost — often worth it for the retrieval layer since it sets the whole system’s ceiling. When: reach for hosted when quality matters more than privacy or cost, or when you lack a GPU. Where: the call shape is the same everywhere — text in, vector out.
# OpenAI embeddings — hosted, high quality, priced per token.
from openai import OpenAI
client = OpenAI()
resp = client.embeddings.create(
model="text-embedding-3-small",
input=["What is your return policy?"],
)
vector = resp.data[0].embedding
print(len(vector)) # 1536Why: query and document vectors are only comparable if they came from the same model — mixing models produces garbage rankings that look plausible but are random. When: pin the model name (and version) in config and re-embed the whole corpus if you ever change it. Where: this is the number-one silent RAG bug — retrieval "works" but returns nonsense after a model swap.
# Pin the embedding model in ONE place and reuse it everywhere.
EMBED_MODEL = "all-MiniLM-L6-v2"
from sentence_transformers import SentenceTransformer
_embedder = SentenceTransformer(EMBED_MODEL)
def embed(texts: list[str]):
return _embedder.encode(texts, normalize_embeddings=True)
# Ingestion and query BOTH call embed(). Change EMBED_MODEL -> re-embed the corpus.