Compatibility checker
Can I run this AI model on my GPU?
Pick a model and a GPU to get an instant, architecture-aware answer: whether it fits in full FP16, needs 8-bit or 4-bit quantization, or requires more than one card. Every estimate accounts for model weights plus the KV cache — the memory term most calculators ignore.
Popular models at a glance
Best precision each model fits at on common GPUs. Tap any cell for the full breakdown.
| Model | RTX 409024 GB | RTX 306012 GB | RTX A600048 GB | A100 80GB80 GB |
|---|---|---|---|---|
| Llama 3.2 3B Instruct | FP16 | FP16 | FP16 | FP16 |
| Llama 3.1 8B Instruct | FP16 | 4/8-bit | FP16 | FP16 |
| Llama 3.1 70B Instruct | No | No | 4/8-bit | 4/8-bit |
| Mistral 7B Instruct v0.3 | FP16 | 4/8-bit | FP16 | FP16 |
| Qwen2.5 7B Instruct | FP16 | 4/8-bit | FP16 | FP16 |
| Qwen2.5 32B Instruct | 4/8-bit | No | 4/8-bit | 4/8-bit |
| Gemma 2 9B Instruct | 4/8-bit | 4/8-bit | FP16 | FP16 |
| Gemma 2 27B Instruct | 4/8-bit | No | 4/8-bit | FP16 |
FP16 = runs at full precision · 4/8-bit = fits with quantization · No = needs a bigger or additional GPU.
How we decide if a model fits
Whether a model runs on a GPU comes down to three memory costs measured against the card's VRAM. First, the model weights: parameter count times bytes per parameter — 2 bytes in FP16, 1 in INT8, and about 0.5 in 4-bit. Second, the KV cache, which stores attention keys and values for every token in the context window and grows linearly with prompt length; it stays in FP16 even when the weights are quantized. Third, a runtime allowance for activations, CUDA context, and memory fragmentation. We add these up at a realistic context length and compare against the GPU, leaving roughly 10% headroom before calling a fit comfortable.
The practical upshot for buyers and builders: quantization is the lever that turns a "no" into a "yes" on consumer cards, but it has limits — a 70B model still needs an 80 GB card even at 4-bit, while an 8B model runs comfortably on a 12 GB card once quantized. For exact numbers at your own context length and batch size, use the VRAM Calculator, and to find the cheapest card that fits a given model, use the GPU Picker.