Guides Local AI

Running LLMs Locally with Ollama: A Beginner's Guide

L
Nejib
• Updated: February 2026

There is something undeniably cool about unplugging your internet cable and still having a conversation with an artificial intelligence that knows how to write Python code.

I travel a lot. Airplane wifi is expensive and terrible. That's why I started messing around with local LLMs. At first, it was a nightmare—compiling C++ files, messing with Python environments, CUDA version mismatches... until **Ollama** came along.

Ollama is basically "Docker for LLMs". It just works. If you have a decent laptop, you can be running Llama 3 in about 5 minutes.

Can my laptop handle it?

This is the #1 question. Here's my rule of thumb based on trying to melt various MacBooks:

  • 8GB RAM: You'll struggle. You can run 7B models (like Llama 3 8B 4-bit quantized), but close your Chrome tabs first. It might be a bit sluggish.
  • 16GB RAM: The sweet spot. You can run 8B models comfortably and even dabble with 14B models (like Qwen).
  • Mac M-Series Chips: Even the base M1 Air runs these models surprisingly well because of the Unified Memory architecture. My M2 Air runs Llama 3 faster than I can read.

The Setup

1. Go to ollama.com.
2. Download functionality.
3. Open your terminal.

Type this one command:

ollama run llama3

You'll see a progress bar as it downloads (it's about 4.7GB). Once it finishes, you'll be dropped into a chat prompt. Say "hi". It replies. No internet needed.

But what about my code?

This is where it gets powerful. Ollama exposes a local API running on `localhost:11434`. You can point your apps to it.

For example, if you use a VS Code extension like "Continue" or "Twinny" for AI coding assistance, you can point it to your local Ollama instance. Now you have GitHub Copilot features, but completely free, and your proprietary code never leaves your machine.

I use this setup when working on NDAs. I can't send client code to OpenAI, but I can send it to my own local Llama 3 instance without breaking any contracts.

Wrap up

Local AI isn't perfect. Llama 3 8B isn't as smart as GPT-4o. It will make mistakes. But it's yours. No rate limits, no monthly bill, no privacy police. Give it a try this weekend.