Setup guide
Run a local LLM on a Raspberry Pi 5.
What to buy, how to set it up in five steps, and what speed to expect, from real, sourced benchmark runs.
Updated 2026-10-17. Every speed links to its source; every price is sourced and dated.
What to buy.
An 8GB Pi 5 is enough for 1–3B models. Add these:
- Raspberry Pi 27W USB-C Power Supply$12
Power
The official 5.1V/5A supply. Raspberry Pi says a standard 5V/3A adapter makes the Pi 5 cap USB peripherals at 600mA. source ↗
$12 at Seeed Studio, price at time of writing (2026-10-17) · Seeed: price source ↗Buy at Seeed StudioBuy on Amazon - Raspberry Pi Active Cooler$5
Cooling
Clip-on heatsink with a temperature-controlled fan, so long generations do not throttle the CPU. source ↗
$5 at Seeed Studio, price at time of writing (2026-10-17) · Seeed: price source ↗Buy at Seeed StudioBuy on Amazon - Samsung PRO Plus 128GB microSDXC$50.99
Storage (pick one)
A2 / U3 card with room for several quantized models.
$50.99 at Samsung US (list price; out of stock when checked), price at time of writing (2026-10-17) · price source ↗Buy on Amazon - SanDisk Extreme 128GB microSDXC
Storage (pick one)
Fast A2 card for the OS and model files.
Buy on Amazon
Pi 5 8GB + these parts, approximate
≈ $242.99
Raspberry Pi 5 8GB $175 + 27W USB-C power supply Seeed: $12 + Active Cooler Seeed: $5 + Samsung PRO Plus 128GB microSD $50.99. Sourced prices at time of writing; they change often, shipping and tax not included.
Want NVMe boot or a case? See the full Pi 5 parts list.
Set it up.
- Flash Raspberry Pi OS (64-bit) to the microSD card with Raspberry Pi Imager. Raspberry Pi Imager ↗
- Fit the Active Cooler and plug in the 27W supply. Boot, connect to your network and update:
sudo apt update && sudo apt full-upgrade - Install Ollama with its official Linux script:
Ollama Linux install ↗curl -fsSL https://ollama.com/install.sh | sh - Run a small model first.
ollama run llama3.2:1b --verbose--verboseprints the eval rate after each answer: that is your generation speed in tokens/sec. - Share your number. Your run is exactly the kind of data this site is built on. Submit it here
What speed to expect.
- 1–2B models: best sourced runs of 11.22 tok/s on Llama 3.2 1B (source ↗) and 11.73 tok/s on Qwen2.5 1.5B (source ↗). That is usable chat speed.
- 3B models: 5.95 tok/s best on Llama 3.2 3B (source ↗). Usable, but slower than you read.
- 8B models: 2.05 tok/s best on Llama 3.1 8B (source ↗). Jeff Geerling measured 2.05 tok/s on the 16GB board, 1.99 tok/s on the 8GB board, so 16GB buys room for bigger models rather than speed.
- Results vary a lot by setup. One paper measured under 1 tok/s on the same models (see the full table below), so check the source before you compare.
Every measured model on the Pi 5.
Best sourced run per model.
| Model | Size | Runtime | Gen tok/s | Source |
|---|---|---|---|---|
| Qwen2.5 1.5B Instruct | 1.5B | Ollama | 11.73 | source ↗ 2026-01-20 |
| Gemma 3 1B | 1B | Ollama | 11.53 | source ↗ accessed 2026-10-17 |
| Qwen2 1.5B | 1.5B | Ollama | 11.29 | source ↗ 2026-01-20 |
| Llama 3.2 1B Instruct | 1B | Ollama | 11.22 | source ↗ accessed 2026-10-17 |
| Qwen2.5-Coder 1.5B | 1.5B | Ollama | 10.26 | source ↗ 2026-01-20 |
| DeepSeek-R1-Distill-Qwen 1.5B | 1.5B | Ollama | 9.04 | source ↗ 2026-01-20 |
| SmolLM2 1.7B Instruct | 1.7B | Ollama | 8.23 | source ↗ accessed 2026-10-17 |
| BitNet b1.58 2B-4T | 2B | Ollama | 8.13 | source ↗ accessed 2026-10-17 |
| Qwen3 1.7B | 1.7B | CPU backend per paper | 7.8 | source ↗ published 2026 |
| Llama 3.2 3B Instruct | 3B | distributed-llama v0.12.3 | 5.95 | source ↗ 2025-02-15 |
| Granite 3.1 Dense 2B | 2B | Ollama | 5.81 | source ↗ accessed 2026-10-17 |
| Qwen2.5 3B | 3B | Ollama | 5.2 | source ↗ accessed 2026-10-17 |
| Vinci Piccolo 1.0 (Qwen3.5-4B based) | 4B | llama.cpp | 3.21 | source ↗ 2026-10-05 |
| Llama 3.1 8B | 8B | Ollama | 2.05 | source ↗ 2025-01-09 |
| Qwen2.5 0.5B Instruct | 500M | Ollama | 1.54 | source ↗ accessed 2026-10-17 |
| Llama 2 13B | 13B | Ollama | 1.35 | source ↗ 2025-01-09 |