Once you have thousands of vectors, a linear scan is too slow. Store and search them with FAISS locally, then see how pgvector and Pinecone do the same job at scale.
Why: comparing a query against every vector is O(n) — fine for hundreds of chunks, too slow for millions, so a vector database indexes them for approximate nearest-neighbor search in milliseconds. When: introduce one the moment your corpus outgrows a quick in-memory loop. Where: it also stores metadata alongside each vector so you can filter and cite.
Brute force: compare query to ALL n vectors -> slow at scale
Vector DB: ANN index (HNSW / IVF) -> top-k in ms -> scales to millions
A vector DB stores: vector + the original text + metadata
(so you can retrieve, filter, and cite)Why: FAISS is a battle-tested library that builds an in-process index — no server, ideal for learning and for small-to-medium corpora. When: use it to prototype retrieval before committing to hosted infrastructure. Where: you keep the texts in a parallel list; FAISS returns integer indices into it.
pip install faiss-cpuWhy: you add all document vectors once, then search returns the indices and distances of the nearest neighbors to your query — that is retrieval. When: normalize vectors and use inner product so the score matches cosine similarity. Where: map the returned indices back to your texts to get the actual passages.
import faiss, numpy as np
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2")
docs = ["Refunds within 30 days.", "Support is 9-5 Mon-Fri.",
"Downloads are non-refundable once accessed."]
vecs = model.encode(docs, normalize_embeddings=True).astype("float32")
index = faiss.IndexFlatIP(vecs.shape[1]) # inner product = cosine on normed vecs
index.add(vecs)
q = model.encode(["can I return a download?"], normalize_embeddings=True).astype("float32")
scores, idx = index.search(q, k=2) # top-2 nearest
for i, s in zip(idx[0], scores[0]):
print(round(float(s), 3), docs[i])Why: production stores add persistence, metadata filtering, and horizontal scale — pgvector keeps vectors in Postgres next to your relational data; Pinecone is a fully managed vector service. When: choose pgvector to avoid new infrastructure, Pinecone (or similar) when you need managed scale. Where: the operations are identical — insert vectors, query nearest — only the client changes.
-- pgvector: a vector column and an index, searched with plain SQL.
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE chunks (
id bigserial PRIMARY KEY,
content text,
source text,
embedding vector(384)
);
CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);
-- Retrieval is a normal query; <=> is cosine distance (smaller = closer).
SELECT content, source
FROM chunks
ORDER BY embedding <=> '[0.02, -0.19, ...]'
LIMIT 5;