Compatibility checker
Can I run this AI model on my GPU?
Pick a model and a GPU to get an instant, architecture-aware answer: whether it fits in full FP16, needs 8-bit or 4-bit quantization, or requires more than one card. Every estimate accounts for model weights plus the KV cache — the memory term most calculators ignore.
Popular models at a glance
Best precision each model fits at on common GPUs. Tap any cell to jump to that model's full per-precision breakdown on the GPU's page.
| Model | RTX 409024 GB | RTX 306012 GB | RTX A600048 GB | A100 80GB80 GB |
|---|---|---|---|---|
| Llama 3.2 3B Instruct | FP16 | FP16 | FP16 | FP16 |
| Llama 3.1 8B Instruct | FP16 | 4/8-bit | FP16 | FP16 |
| Llama 3.1 70B Instruct | No | No | 4/8-bit | 4/8-bit |
| Mistral 7B Instruct v0.3 | FP16 | 4/8-bit | FP16 | FP16 |
| Qwen2.5 7B Instruct | FP16 | 4/8-bit | FP16 | FP16 |
| Qwen2.5 32B Instruct | 4/8-bit | No | 4/8-bit | 4/8-bit |
| Gemma 2 9B Instruct | 4/8-bit | 4/8-bit | FP16 | FP16 |
| Gemma 2 27B Instruct | 4/8-bit | No | 4/8-bit | FP16 |
| Qwen2.5 3B Instruct | FP16 | FP16 | FP16 | FP16 |
| Qwen2.5 14B Instruct | 4/8-bit | 4/8-bit | FP16 | FP16 |
| Qwen2.5 72B Instruct | No | No | 4/8-bit | 4/8-bit |
| Qwen2.5 Coder 7B Instruct | FP16 | 4/8-bit | FP16 | FP16 |
| Qwen2.5 Coder 32B Instruct | 4/8-bit | No | 4/8-bit | 4/8-bit |
| Qwen3 8B | FP16 | 4/8-bit | FP16 | FP16 |
| Qwen3 14B | 4/8-bit | 4/8-bit | FP16 | FP16 |
| Qwen3 32B | 4/8-bit | No | 4/8-bit | 4/8-bit |
| DeepSeek R1 Distill Qwen 7B | FP16 | 4/8-bit | FP16 | FP16 |
| DeepSeek R1 Distill Qwen 14B | 4/8-bit | 4/8-bit | FP16 | FP16 |
| DeepSeek R1 Distill Qwen 32B | 4/8-bit | No | 4/8-bit | 4/8-bit |
| DeepSeek R1 Distill Llama 8B | FP16 | 4/8-bit | FP16 | FP16 |
| Phi-3.5 Mini Instruct | FP16 | FP16 | FP16 | FP16 |
| Phi-4 | 4/8-bit | 4/8-bit | FP16 | FP16 |
| Mistral Nemo Instruct | 4/8-bit | 4/8-bit | FP16 | FP16 |
| SmolLM2 1.7B Instruct | FP16 | FP16 | FP16 | FP16 |
| Qwen2.5 0.5B Instruct | FP16 | FP16 | FP16 | FP16 |
| Qwen2.5 1.5B Instruct | FP16 | FP16 | FP16 | FP16 |
| Qwen2.5 Coder 1.5B Instruct | FP16 | FP16 | FP16 | FP16 |
| Qwen2.5 Coder 14B Instruct | 4/8-bit | 4/8-bit | FP16 | FP16 |
| Qwen3 0.6B | FP16 | FP16 | FP16 | FP16 |
| Qwen3 1.7B | FP16 | FP16 | FP16 | FP16 |
| Qwen3 4B | FP16 | FP16 | FP16 | FP16 |
| QwQ 32B | 4/8-bit | No | 4/8-bit | 4/8-bit |
| DeepSeek R1 Distill Qwen 1.5B | FP16 | FP16 | FP16 | FP16 |
| DeepSeek R1 Distill Llama 70B | No | No | 4/8-bit | 4/8-bit |
| Mistral Small 24B Instruct | 4/8-bit | No | 4/8-bit | FP16 |
| Granite 3.1 8B Instruct | FP16 | 4/8-bit | FP16 | FP16 |
| OLMo 2 7B Instruct | FP16 | 4/8-bit | FP16 | FP16 |
| SmolLM2 360M Instruct | FP16 | FP16 | FP16 | FP16 |
FP16 = runs at full precision · 4/8-bit = fits with quantization · No = needs a bigger or additional GPU.
Browse by GPU
Pick your card to see every model it runs, and the precision each one needs.
Consumer
Workstation
Data center
How we decide if a model fits
Whether a model runs on a GPU comes down to three memory costs measured against the card's VRAM. First, the model weights: parameter count times bytes per parameter — 2 bytes in FP16, 1 in INT8, and about 0.5 in 4-bit. Second, the KV cache, which stores attention keys and values for every token in the context window and grows linearly with prompt length; it stays in FP16 even when the weights are quantized. Third, a runtime allowance for activations, CUDA context, and memory fragmentation. We add these up at a realistic context length and compare against the GPU, leaving roughly 10% headroom before calling a fit comfortable.
The practical upshot for buyers and builders: quantization is the lever that turns a "no" into a "yes" on consumer cards, but it has limits — a 70B model still needs an 80 GB card even at 4-bit, while an 8B model runs comfortably on a 12 GB card once quantized. For exact numbers at your own context length and batch size, use the VRAM Calculator, and to find the cheapest card that fits a given model, use the GPU Picker.