Running Large Language Models (LLMs) locally used to require complex Python environments, custom PyTorch dependencies, and manually managed CUDA drivers. Ollama changed that completely. It operates like a package manager for open-weight AI models, allowing you to run frontier models entirely offline on your Mac or Windows PC with a single terminal command.
In this step-by-step guide, we will cover minimum hardware requirements, step-by-step installation for macOS and Windows, essential CLI commands, performance optimization, and how to hook your local setup into a ChatGPT-style browser UI.
Tested Hardware Environment for this Guide:
- Mac Test Machine: Apple M3 Max (36GB Unified Memory), macOS Sonoma
- Windows Test Machine: Intel i7-13700K, 32GB DDR5 RAM, NVIDIA RTX 4070 (12GB VRAM), Windows 11
- Ollama Runtime Version: v0.5+
1. Hardware Requirements: Will Your Machine Run Local Models?
Ollama is lightweight, but the models themselves demand memory. The most common mistake beginners make is attempting to run a model larger than their machine’s available RAM or VRAM, resulting in system freezes or crawl-paced output.
On Apple Silicon (M1/M2/M3/M4), macOS shares memory dynamically between the CPU and GPU (Unified Memory). On Windows, Ollama will run much faster if the entire model fits inside your NVIDIA/AMD dedicated VRAM, though it can offload remaining parameters to system RAM at a performance penalty.
| Model Parameter Size | Required VRAM / RAM | Recommended Model | Best For |
|---|---|---|---|
| 3 Billion (3B) | 4 GB – 8 GB | llama3.2:3b | Laptops, older hardware, basic summaries |
| 7B – 8 Billion | 8 GB – 12 GB | llama3.1:8b / qwen2.5-coder:7b | General reasoning, coding, daily tasks |
| 14B – 32 Billion | 16 GB – 32 GB | qwen2.5:14b / command-r | Complex logic, detailed creative writing |
| 70 Billion (70B) | 64 GB+ Unified RAM / Dual GPUs | llama3.3:70b | Enterprise-grade analysis & complex coding |
2. How to Install Ollama on macOS
Installing Ollama on macOS takes less than two minutes. It runs as a background service and automatically leverages Metal GPU acceleration on Apple Silicon without manual configuration.
Option A: Standalone Installer (Recommended)
- Navigate to the official download page at ollama.com/download and select macOS.
- Download the
Ollama-darwin.zipfile and open it. - Drag the Ollama app into your Applications folder.
- Launch Ollama from Applications. Click Open when macOS prompts security authorization.
- Follow the quick terminal setup wizard to install CLI tools.
Option B: Homebrew Installation
If you prefer using package managers, open your Terminal and run:
brew install ollama3. How to Install Ollama on Windows
Ollama natively supports Windows 10 and Windows 11 (64-bit). It automatically detects NVIDIA CUDA drivers and AMD ROCm support.
- Visit ollama.com/download/windows and click Download for Windows.
- Run the downloaded
OllamaSetup.exefile. - Click Install. Once completed, Ollama will automatically start and sit quietly in your system tray (near the clock).
- Open Command Prompt or PowerShell and verify installation:
ollama --versionTroubleshooting Note: If Command Prompt returns “command not found”, restart your PC. The Windows system PATH environment variable needs a system reboot to index new executable binaries.
4. Pulling and Running Your First Model
Once Ollama is installed, you pull and run models directly through your terminal using simple CLI commands.
Step 1: Download and Chat with Meta’s Llama 3.1 (8B)
Open your Terminal (macOS) or PowerShell (Windows) and type:
ollama run llama3.1Ollama will automatically download the quantized model weights (~4.7 GB). Once the download completes, an interactive prompt will open:
>>> Send a message (Press Ctrl+D or type /bypass to exit)
>>> Write a 3-sentence summary explaining quantum computing to a 10-year-old.To end the session and unload the model from active memory, type /bye or hit Ctrl + D.
Step 2: Key Ollama CLI Commands Cheatsheet
| Command | Action / What it does |
|---|---|
ollama list | Shows all models currently downloaded on your drive. |
ollama pull <model-name> | Downloads a model without launching an interactive session immediately. |
ollama ps | Displays which models are loaded in VRAM right now. |
ollama stop <model-name> | Forces a running model to unload from memory instantly. |
ollama rm <model-name> | Deletes the specified model from your hard drive to free up space. |
5. Crucial Optimization: Change Model Storage Directory
By default, Ollama saves models in your OS primary boot drive:
- macOS:
~/.ollama/models - Windows:
C:\Users\<Username>\.ollama\models
Because AI models consume 4 GB to 40 GB each, your main drive will fill up fast. We highly recommend moving storage to a secondary SSD drive before downloading multiple models.
On Windows:
- Search for “Edit environment variables for your account” in the Start Menu.
- Under User variables, click New.
- Set Variable name:
OLLAMA_MODELS - Set Variable value:
D:\AI_Models\Ollama(or your preferred SSD directory). - Click OK, then right-click the Ollama icon in your Windows system tray and select Quit. Relaunch Ollama.
On macOS:
Open Terminal and set the environment variable using launchctl:
launchctl setenv OLLAMA_MODELS "/Volumes/ExternalSSD/OllamaModels"Quit Ollama completely from the Mac menu bar and re-open it.
6. Adding a Web UI Interface (Open WebUI Setup)
Interacting with models in the command line is great for testing, but most users prefer a modern graphical user interface similar to ChatGPT. The best open-source dashboard available is Open WebUI.
[ Visual Diagram Setup ]
Browser (Open WebUI: Port 3000) <── Local API Bridge ──> Ollama Engine (Port 11434) <── Local System VRAM
Step 1: Install Docker Desktop
Open WebUI deploys cleanly via Docker. Download and install Docker Desktop for Mac or Windows from docker.com if you haven’t already.
Step 2: Run Open WebUI Container
With Docker Desktop running, paste the following command into your Terminal or PowerShell:
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:mainStep 3: Access Your Local Workspace
- Open your web browser and navigate to
http://localhost:3000. - Click Sign Up to create your local admin account (your credentials stay strictly on your device).
- Select any downloaded Ollama model from the top dropdown menu and begin chatting.
7. Real-World Troubleshooting Guide
Issue 1: “Connection Refused” when launching Open WebUI
Cause: The Ollama background service is not running or is blocked from local API calls.
Fix: Ensure Ollama is running in your system tray/menu bar. If running manually, execute ollama serve in a separate terminal window.
Issue 2: Generation speed is painfully slow (less than 2 tokens/sec)
Cause: Your model exceeds available VRAM, forcing CPU offloading.
Fix: Run ollama ps while prompting the model. If the GPU percentage indicates partial offload (e.g., 50% CPU / 50% GPU), switch to a smaller model (such as moving from an 8B model to a 3B model) or a higher quantization format.
Manish Prakash Dubey is an AI educator and technology writer based in India. He founded WiseAIWorld to make artificial intelligence simple and practical for students, professionals, and beginners. His work focuses on AI basics, machine learning, deep learning, NLP, computer vision, and real-world AI tools.
