How to read your results

Two lists. Runs on your pool is largest-model-first, each showing the highest-quality quantization that fits — a model appearing at Q2_K is a real quality trade-off, not a free win. Out of reach shows the nearest misses and exactly what each would need. The utilisation percentage tells you how much headroom is left for longer context.

What can each VRAM tier run?

6 GB — RTX 2060 6GB and similar

Entry tier. 4B-class models run comfortably: Gemma 4 Edge 4B fits at Q8_0 (near-lossless) with room to spare. 7–8B models fit at lower quantization with short context.

8 GB — RTX 3050, 3060 Ti, 3070, 4060, 4060 Ti 8GB, 5060, Arc A750, RX 7600, RX 580 8GB

The most common tier in the wild. Comfortable home of Llama 3.1 8B, Qwen3 8B and Mistral 7B, with Phi-4 14B reachable at lower quantization. Quantized KV cache extends this to longer context or slightly larger models.

10–12 GB — RTX 3080 10GB, RTX 3080 12GB, RTX 2060 12GB, RTX 3060 12GB, RTX 5070, Arc B580, RX 7700 XT, RX 6700 XT

The sweet spot for Gemma 4 12B and Qwen2.5 14B, and the entry point for 30B-class Mixture-of-Experts models such as Qwen3 30B-A3B, where only a fraction of parameters are active per token.

16 GB — RTX 4060 Ti 16GB, 4070 Ti Super, 5060 Ti 16GB, 5070 Ti, 5080, Arc A770 16GB, RX 7800 XT, RX 9070 XT, RX 6800 XT

Where serious local work starts. Qwen3 32B and Gemma 4 31B become viable at Q3–Q4, and Gemma 4 26B MoE fits comfortably.

24 GB — RTX 3090, 3090 Ti, 4090, RX 7900 XTX

The enthusiast standard. 32B-class dense models at good quantization, 70B-class at aggressive quantization, and — thanks to sliding-window attention combined with quantized KV cache — genuinely long context on a single card.

32 GB and above — RTX 5090, multi-GPU pools

70B-class models comfortably; multi-GPU pools reach the 100B+ tier.

The three settings that change the answer most

1. Quantization (model weights)

The biggest single lever. Q4_K_M is the community default and roughly quarters FP16 memory. We use effective bits-per-weight — Q4_K_M is 4.85 bpw, not the theoretical 4.0 most tools assume, because K-quants promote attention layers to higher precision.

2. Context length

The cost people forget. The KV cache grows with context and can exceed the model weights themselves at long context. Crucially, not all architectures pay the same price: DeepSeek and Kimi use MLA compression, and Gemma 4 uses hybrid sliding-window attention where only every sixth layer caches the full context. Tools that treat every model as full-attention overstate these by three times or more.

3. KV cache precision

The long-context unlock, set with -ctk and -ctv in llama.cpp. Dropping the cache from F16 to Q8_0 roughly halves it; Q4_0 cuts it to a little over a quarter. On a 24 GB card at 128k context this is the difference between a handful of runnable models and most of the catalog. It is not free — quantized KV cache is lossy, and asymmetrically so, with the V cache degrading more than the K cache — but it is the single most effective long-context lever on consumer hardware. See our llama.cpp performance flags guide for the measured trade-offs.

Why the exact GPU variant matters

The same card name ships with different amounts of VRAM: RTX 4060 Ti 8GB vs 16GB, RTX 3080 10GB vs 12GB, RTX 2060 6GB vs 12GB, Arc A770 8GB vs 16GB. That gap decides whether a 14B model fits. Hardware auto-detectors routinely miss the distinction — you don't, because you pick your card by name.

What this tool does not tell you

It computes memory, not speed and not quality. It will not tell you tokens per second — that depends on memory bandwidth, and we own no hardware to benchmark on, so we do not publish numbers we have not measured. It will not tell you how much quality a low quantization costs. And a model that "fits" at 99% utilisation will be fragile in practice. For what happens when a model does not fit and you stream it from disk anyway, see our streaming-weights reality check.

Frequently asked questions

What LLM can I run on my GPU?

It depends on your VRAM. Pick your card above for the exact list. Broadly: 8 GB runs 7–8B models comfortably, 12 GB reaches 14B and 30B-class MoE, 16 GB handles 32B at reduced quantization, and 24 GB runs 32B comfortably and 70B at aggressive quantization.

What AI model can my GPU run if I only have 8 GB of VRAM?

7–8B dense models such as Llama 3.1 8B, Qwen3 8B and Mistral 7B at Q4–Q5, plus smaller Mixture-of-Experts models. Enabling quantized KV cache extends this to longer context or slightly larger models.

How is this different from GPU auto-detect tools?

It uses exact VRAM math — effective bits-per-weight, architecture-correct KV cache including MLA compression and sliding-window attention, and real backend overhead — instead of lookup tables or letter grades. It also distinguishes VRAM variants of the same card, which auto-detection frequently gets wrong.

Which AI software works with my GPU?

The tool models llama.cpp and GGUF-class runtimes (llama.cpp, Ollama, LM Studio) as well as vLLM and ExLlamaV2, covering NVIDIA, AMD and Intel hardware. Your backend choice changes overhead slightly; it does not change the underlying memory math.

Does quantizing the KV cache hurt quality?

Yes, though modestly at Q8_0, which community practice treats as safe for general use. Q4_0 saves more but carries real quality risk, especially on the V cache. The tool reports the memory saving; the quality trade-off is yours to judge.

Why does the same model need different VRAM at different context lengths?

Because the KV cache grows with context, and at long context it can exceed the model weights. Architectures differ: models using MLA compression or sliding-window attention pay far less than standard full-attention models.

Can I run a 70B model on 24 GB?

Only at aggressive quantization with short context, and quality suffers noticeably. If it does not fit your settings, the tool shows it under "out of reach" with the exact requirement.

Is it free?

Yes. Free, no signup, no hardware scan. Everything runs in your browser and nothing about your machine leaves your device.