| GPU | LLM Score | Price | VRAM | Est. tokens/sec | TDP | Model fit | Compare |
|---|
Where the speed figures come from
Every tokens/sec figure on this page is for the same test: Llama 3.1 8B, Q4_K_M quantisation, single-batch inference. Mixing results from different models or quantisation levels is the most common way these comparisons go wrong, so only one scale is used here. Speeds move with driver and runtime versions — treat them as a ranking, not a guarantee.
No marker — measured. Published benchmark results for that exact card and test: RTX 5090, RTX 4090, RTX 4080 Super, RTX 3090, RX 7900 XTX and Intel Arc B580. Source: CompareAIHardware, which marks cards it has not measured rather than filling the gap with an estimate.
† calibrated. The RTX 4070 Super and RTX 4060 Ti 16GB have published figures for a 7B model rather than 8B. Two cards appear in both sets — the RTX 4090 at 135 against 125, and the RTX 3090 at 95 against 85 — giving a conversion of about 1.10 between the two tests. That ratio is applied to bring the 7B numbers onto this page's 8B scale. Source: mustafa.net llama.cpp benchmarks.
* estimated. Two cards have no usable published figure for this test. Their speeds are derived from memory bandwidth, which is what limits single-batch inference on a model this size. Across the measured NVIDIA 40 and 50 series cards the ratio is about 0.131 tok/s per GB/s, putting the RTX 4070 Ti Super near 88. The Intel Arc A770 is shown at 38 — below what its 560 GB/s alone would suggest, because its older Alchemist driver stack does not reach the efficiency the newer B580 achieves. That last adjustment is a judgement, not a measurement, and it is the least certain number on this page.
One figure was found and deliberately not used: a published 96.3 tok/s for the RTX 4070 Ti Super on Qwen3 8B, from a source that also puts the RTX 4090 at 93% of that card's speed. A RTX 4090 has 1008 GB/s of memory bandwidth against the 4070 Ti Super's 672 GB/s, so that ordering is not credible and the result was discarded rather than averaged in.
Where the prices come from
Prices refresh daily from the same feed /parts/ uses, matched on model name and VRAM so a 16GB card can never be priced off its 8GB sibling. A figure marked * not live did not come from that feed today.
Cards without a live price today are listed here once prices load.
Baseline AI PC build
Local LLMs are GPU-first, but the rest of the PC still matters. A sensible starter build for one GPU is:
- CPU: modern 6-core or better Ryzen 5 / Core i5
- RAM: 32GB minimum, 64GB preferred for 30B+ models or multitasking
- Storage: 1TB NVMe so model downloads do not crowd Windows
- PSU: 650W for midrange GPUs, 850W+ for RTX 3090/4090 class cards
- Cooling: airflow case, especially for used 300W+ GPUs
How the score works
The StashGrid LLM score is intentionally different from a gaming benchmark. It weights VRAM/model fit first, then speed per dollar, power efficiency, and setup friendliness. A cheap 16GB card can beat a faster 12GB card when the model you want will not comfortably fit.
Formula: 45% model fit, 30% speed per dollar, 15% watts, 10% software setup. It is a buying guide, not a lab benchmark.
How much VRAM do you need?
The model has to fit in VRAM with room left over for the conversation. These are the download sizes of the Q4_K_M versions most guides start with, taken from the published model files:
- 8B models (Llama 3.1 8B): 4.9GB. Fits every card on this page, and 8GB cards too.
- 14B models (Qwen2.5 14B): 9.0GB. Squeezes onto a 12GB card with a short context. 16GB is comfortable.
- 32B models (Qwen2.5 32B): 19.9GB. Needs a 24GB card.
- 70B models (Llama 3.1 70B): 42.5GB. Does not fit on any single consumer card. A 32GB card holds most of it and the rest runs from system RAM, which works but is far slower. That is what “70B split/offload” means in the filter above.
Long conversations and pasted documents need more on top, because the memory that holds the context grows with every token kept. Leave a few GB spare if you work that way.
Best GPU for Ollama: which current models fit which card
These are the default downloads from the Ollama library, checked 18 October 2026. Most are 4-bit (Q4_K_M) files. A model runs at full speed only when the whole file plus its context memory fits in VRAM. Leave roughly 2 GB spare for that, and more for long chats.
| Model (Ollama tag) | Download | Card size that holds it | Cards on this page |
|---|---|---|---|
| llama3.1:8b, qwen3:8b | 4.9–5.2 GB | 8 GB | All ten |
| gemma3:12b | 8.2 GB | 12 GB | All ten |
| qwen3:14b, phi4, deepseek-r1:14b | 9.0–9.3 GB | 12 GB (tight), 16 GB comfortable | All ten; the 12 GB cards only with a short context |
| gpt-oss:20b | 14 GB | 16 GB | Every 16, 24 and 32 GB card |
| mistral-small3.2 (24B) | 15 GB | 24 GB | RTX 3090, RTX 4090, RX 7900 XTX, RTX 5090 |
| gemma3:27b | 17 GB | 24 GB | RTX 3090, RTX 4090, RX 7900 XTX, RTX 5090 |
| qwen3:30b (mixture of experts) | 19 GB | 24 GB | RTX 3090, RTX 4090, RX 7900 XTX, RTX 5090 |
| qwen3:32b, deepseek-r1:32b | 20 GB | 24 GB, with a shorter context | RTX 3090, RTX 4090, RX 7900 XTX, RTX 5090 |
| llama3.1:70b, deepseek-r1:70b | 43 GB | No single consumer card | Split between GPU and system RAM: works, but far slower |
| gpt-oss:120b | 65 GB | No consumer card | n/a |
The context catch. Recent versions of Ollama choose a default context length from your VRAM: 4k tokens on cards under 24 GB, 32k on cards with 24–48 GB. That context needs memory of its own.
- Qwen3 14B: about 160 KB per token, so roughly 0.6 GB at 4k and 5 GB at 32k.
- Qwen3 32B: about 256 KB per token, so roughly 8 GB at 32k. A 20 GB model plus 8 GB of context does not fit in 24 GB, so Ollama moves part of it to system RAM and it slows down.
The fix: lower the context (/set parameter num_ctx 8192 inside ollama run), or set OLLAMA_KV_CACHE_TYPE=q8_0, which roughly halves that memory with a very small loss in precision.
Check it in one command. Run ollama ps while the model is loaded. The PROCESSOR column should read 100% GPU. A split such as 48%/52% CPU/GPU means part of the model is running from system RAM.
What each VRAM size gets you
- 12 GB (RTX 4070 Super, Arc B580): 8B models with room for long chats; 12–14B models at Ollama's 4k default.
- 16 GB (RTX 4060 Ti 16GB, RTX 4070 Ti Super, RTX 4080 Super, Arc A770): 14B models comfortably. gpt-oss:20b, which Ollama says runs in as little as 16 GB.
- 24 GB (RTX 3090, RTX 4090, RX 7900 XTX): the 24–32B class, the biggest step up in quality on a single card. Watch the 32k default context with 32B models (see above).
- 32 GB (RTX 5090): 32B models with long context. Still not enough for a 70B model on its own.
Quick recommendations
Best low-risk starter: RTX 4060 Ti 16GB if buying new and power matters. Best used value: RTX 3090 24GB if you accept used-card risk, heat, and power draw. Best no-compromise consumer card: RTX 4090 or newer 32GB-class cards when budget is secondary. Best AMD caveat: RX 7900 XTX has excellent VRAM per dollar, but local AI setup can require more checking than CUDA-based NVIDIA paths.
Sources and trust notes
- Ollama model library for local model families and model size context.
- llama.cpp for local inference backend support including CUDA, Metal, Vulkan, and CPU paths.
- LM Studio docs for consumer local LLM workflow context.
- Compute Market LLM inference benchmark roundup and GPU Hunter LLM GPU guide for public speed/value reference points. Treat tokens/sec as estimates, not guarantees.
FAQ
What matters most for running local LLMs?
VRAM usually matters first because the model weights and context need to fit on the GPU. After fit, memory bandwidth, tensor performance, software support, RAM, CPU, and power supply determine the experience.
Can I run a local LLM without a GPU?
Yes, CPU-only inference is possible, but it is much slower. A GPU is strongly preferred for interactive chat, coding assistants, and larger models.
How much VRAM do I need for local AI?
8GB can run small quantized models, 12GB is a practical starter tier, 16GB is more comfortable, and 24GB or more is better for larger 30B class models or longer context.
Is NVIDIA required for local LLMs?
No, but NVIDIA CUDA support is usually the easiest path. AMD and Intel can work through tools and backends such as Vulkan, ROCm, or vendor-specific runtimes, but setup and model support may vary.
Is VRAM more important than raw GPU speed?
For local LLMs, yes in many cases. If the model does not fit in VRAM, performance drops sharply or the model will not run comfortably. Once it fits, speed and memory bandwidth matter more.
Can two GPUs combine VRAM?
Some tools and backends can split a model across multiple GPUs, but it is more complex than a single large-VRAM card. For most buyers, one 24GB card is easier than two smaller cards.
Should I buy a prebuilt AI PC?
Prebuilt systems can be a good option if they include a real GPU, adequate PSU, 32GB to 64GB RAM, and clear cooling. Avoid vague listings that say "AI ready" without VRAM, PSU, and GPU model details.
What is the best GPU for Ollama?
It depends on the largest model you want to run. Start with the table above: a 16 GB card such as the RTX 4060 Ti 16GB runs 14B models and gpt-oss:20b. A 24 GB card such as a used RTX 3090 runs the 27–32B class. No single consumer card holds a 70B model.
Is 12GB of VRAM enough for local AI?
For 8B models, yes, with room for long conversations. For 12–14B models it works at Ollama's default 4k context, but it is tight. For 20B and up you need 16 GB or more.