Workstation · NVIDIA · 48 GB VRAM

What can you run on the RTX A6000?

The RTX A6000 has 48 GB of VRAM. Of the 38 models we track, 38 run on this card — 28 at full FP16 precision and 10 once quantized. Every figure below counts model weights plus the KV cache at a 4,096-token context.

Runs at full FP16 precision

These models fit without quantization, so you keep full output quality.

Runs with quantization

These need an INT8 or 4-bit build (GPTQ, AWQ, or GGUF). Quality stays close to full precision for most chat work, but re-test structured output and tool calling before shipping.

How to read these numbers

Whether a model runs on the RTX A6000 comes down to three memory costs measured against its 48 GB. First the weights: parameter count times bytes per parameter — 2 bytes in FP16, 1 in INT8, roughly 0.5 at 4-bit. Second the KV cache, which holds attention keys and values for every token in the context window and grows linearly with prompt length; it stays in FP16 even when the weights are quantized. Third, an allowance for activations, CUDA context, and fragmentation. A fit is only called comfortable when the total leaves about 10% headroom.

The practical consequence is that the table above is a starting point, not a guarantee. A model listed as fitting at 4,096 tokens can still run out of memory once conversations get long or several requests run at once, because the KV cache term grows with both. If you plan to use long context or serve concurrent users, size with the VRAM Calculator at your real context length before committing.

Compare other workstation GPUs

Frequently asked questions