You do not have to call a paid API. Run open models on your own machine with Ollama, pull models from Hugging Face, and know when hosted beats local and vice versa.
Why: hosted APIs (Anthropic, OpenAI, Gemini) trade money for zero ops and frontier quality; local models trade setup for privacy, no per-token cost, and offline use. When: reach for local when data cannot leave your machine or you are iterating cheaply; reach for hosted when you need the strongest model with no infrastructure.
HOSTED API LOCAL (Ollama / HF)
cost per token free after download
privacy data leaves your box stays on your machine
quality frontier models smaller, catching up
ops none you run the server / GPU
latency network round-trip local, but GPU-bound
Most teams use both: a local model for cheap bulk work,
a hosted model for the hard requests.Why: Ollama downloads an open model and serves it behind a local API with one command — no GPU wrangling. When: use it for offline development, private data, or avoiding API bills while you prototype. Where: it listens on localhost:11434 and speaks a simple HTTP API.
# Install from ollama.com, then:
ollama pull llama3.2 # download the model once
ollama run llama3.2 "Say hi" # chat from the terminal
# It also serves an HTTP API on localhost:11434:
curl http://localhost:11434/api/generate -d '{
"model": "llama3.2",
"prompt": "Explain embeddings in one line.",
"stream": false
}'Why: Ollama exposes an OpenAI-compatible endpoint, so the same client code targets a local model just by changing base_url — your app stays provider-agnostic. When: swap base_url and model to move a feature between local and hosted without rewriting it.
from openai import OpenAI # pip install openai
client = OpenAI(
base_url="http://localhost:11434/v1", # your local Ollama
api_key="ollama", # required by the SDK, ignored locally
)
response = client.chat.completions.create(
model="llama3.2",
messages=[{"role": "user", "content": "Name three uses for local LLMs."}],
)
print(response.choices[0].message.content)Why: Hugging Face is the registry — the "npm" of models — where open models (including the ones Ollama pulls) and datasets live. When: browse the Hub to pick a model by task, license, and size; use the transformers library when you need full control over inference. Where: model names like "meta-llama/Llama-3.2-1B" are Hub coordinates.
# pip install transformers torch
from transformers import pipeline
# Downloads the model from the Hugging Face Hub on first run, then runs
# it locally. Great for small task-specific models (classification, NER).
classifier = pipeline("sentiment-analysis")
print(classifier("This course is exactly what I needed."))
# [{'label': 'POSITIVE', 'score': 0.9998}]