Ollama lets you run open-weight language models on your own computer. Once a model is downloaded, the chat works without an internet connection, and your prompts are processed locally. This guide shows how to install it, which model fits your hardware, and how to call it from your own tools.
1. Install Ollama
Download the installer for macOS, Windows or Linux from ollama.com and run it. Then open a terminal.
2. Pick a model for your hardware
The model size is the main thing to get right. Rough guide, for models quantized to 4 bits (Ollama's default for most tags):
- 8 GB RAM: small models such as
lfm3:1.7b,llama3.2(3B) or the smallestgemma4variants. Close other heavy apps first. - 16 GB RAM: 7B to 8B models such as
qwen3orgemma4at mid sizes.gpt-oss:20bis listed for 16 GB machines. - 32 GB RAM or a dedicated GPU: larger models up to about 30-40B, such as
qwen3.8(27B, the most popular local model this year) orflint:40bfor coding (a MoE with only 2.5B active parameters, so it's fast). - 64 GB+ or a workstation GPU:
gpt-oss:120bornemotron-3.5-super:120b.
The model library changes often. Check ollama.com/library
for current names and sizes. Older models such as llama3 are still there but have newer, smaller alternatives.
3. Run your first model
ollama run llama3.2
The first run downloads the model, then opens a chat prompt. Use ollama pull <model> if you only want to download it
without starting a chat.
4. Use it from your code
Ollama runs a local server on http://localhost:11434, with an OpenAI-compatible endpoint at /v1:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") # key is ignored locally
reply = client.chat.completions.create(
model="llama3.2",
messages=[{"role": "user", "content": "Write a haiku about local AI."}],
)
print(reply.choices[0].message.content)
5. Use it with Claude Code or Codex
Ollama also speaks Anthropic's API, so Claude Code can run on a local model: set
ANTHROPIC_BASE_URL=http://localhost:11434, ANTHROPIC_AUTH_TOKEN=ollama and
ANTHROPIC_MODEL=qwen3.8, or simply run ollama launch claude. For Codex, point a
provider at http://localhost:11434/v1. More in our coding guide.
6. Connect an editor
Extensions such as Continue and Twinny (VS Code) can point at the local server, so autocomplete and chat use your own model. Code sent to a local model stays on your machine, which matters if your contract or employer restricts sending code to outside services. Check your own policy before relying on that.
Limits to expect
Small local models are noticeably weaker than the best hosted models, especially on long reasoning and tricky code. Local speed depends on your RAM, GPU and model size. Run a few prompts from your real work before deciding.
Last checked: 4 November 2026 (Ollama 0.40, which uses Apple's MLX by default on Apple Silicon). Model names and sizes come from the Ollama library page; check it for updates.