Measured on our own hardware
The 16 GB Local AI Playbook
Every number here is measured on one real card — an NVIDIA GeForce RTX 5060 Ti with 16 GB of memory — not estimated from a spec sheet. We own exactly one consumer GPU, so instead of guessing at hardware we don't have, we go deep on the tier most people actually run: what fits in 16 GB, at what speed, at the quant a buyer would use.
Last measured August 10, 2026 UTC · driver 580.173.02 · Ollama 0.23.2
What fits your card?
Speed is measured on the 16 GB card above. Whether a model fits is mostly its memory footprint, which travels between cards far better than its speed does — so filter by the VRAM you have and see which of these measured models fit inside it.
The measured ladder
| Model | Params | Quant | VRAM used | On GPU | Context | Generation | Prompt | Cold load |
|---|---|---|---|---|---|---|---|---|
Gemma 3 · 1Bollama pull gemma3:1b | 999.89M | Q4_K_M | 1.3 GB | 100% GPU | 32768 | 263 tok/s | 7931 tok/s | 2.61s |
Qwen2.5 · 3B Instructollama pull qwen2.5:3b-instruct | 3.1B | Q4_K_M | 3.1 GB | 100% GPU | 32768 | 175 tok/s | 9768 tok/s | 2.66s |
Gemma 3 · 4Bollama pull gemma3:4b | 4.3B | Q4_K_M | 4.6 GB | 100% GPU | 131072 | 120 tok/s | 3697 tok/s | 3.14s |
Llama 3.2 · 3Bollama pull llama3.2:3b | 3.2B | Q4_K_M | 4.8 GB | 100% GPU | 131072 | 174 tok/s | 8985 tok/s | 2.92s |
Qwen2.5 · 7B Instructollama pull qwen2.5:7b-instruct | 7.6B | Q4_K_M | 6.3 GB | 100% GPU | 32768 | 88 tok/s | 4960 tok/s | 8.55s |
Qwen2.5 · 14Bollama pull qwen2.5:14b | 14.8B | Q4_K_M | 13 GB | 100% GPU | 32768 | 45 tok/s | 2548 tok/s | 9.97s |
No measured model fits that budget on its own.
Where to start
- On a 16 GB card, start with Qwen2.5 · 14B — the largest measured model that fits, at 13 GB and 45 tok/s.
- Want it snappier? Drop to Qwen2.5 · 7B Instruct at 6.3 GB and 88 tok/s — about 2× the generation speed.
- On an 8 GB card, run Llama 3.2 · 3B — 4.8 GB and 174 tok/s.
Rather not choose by hand? shadowfetch-llm does the same job from the command line — it measures your card and picks the largest model that fits.
How these were measured
- One NVIDIA GeForce RTX 5060 Ti (16 GB). Generation rate is eval tokens ÷ eval time from the Ollama API, taken as the median of 3 timed runs of 200 tokens after a warm-up.
- VRAM used and “on GPU” are what
ollama psreports for the resident model — the model's real footprint, and whether all of it sits on the GPU or spills to the CPU (which is the moment speed falls off a cliff). - Params, quantisation and architecture are read from
ollama show— the model file's own metadata, not typed by us. Everything shown is Q4_K_M-class weights, the quant most people actually run. - Speed is specific to this card; your tokens/sec will differ on other hardware. Footprint and fit travel much better — that's why the filter above keys off memory, not speed.