Cost tool
LLM inference cost calculator
Three ways to serve an open model — a managed API, a rented cloud GPU, or your own hardware — priced side by side over three years. The comparison includes the costs teams usually forget: setup, maintenance, power, and the engineering time to keep it running.
Your deployment profile
Pick the model you plan to serve and the traffic you expect. Hardware cost is driven by the model's memory footprint; API cost is driven by token volume.
Needs about 167.2 GB in FP16 — this sets which GPU tier you are paying for.
Input plus output tokens across all requests in a month.
Self-hosted and cloud GPUs bill whether or not they are busy. Fewer hours favors an API.
Used to flag license thresholds that apply at scale.
Total Cost of Ownership
API vs Cloud GPU vs Self-Hosted cost comparison
Cost estimates are approximate and vary by region, usage patterns, and provider.
| Period | API | Cloud GPU | Self-Hosted |
|---|---|---|---|
| Year 1 | $125 | $47,907 | $81,828 |
| Year 2 | $125 | $47,407 | $49,028 |
| 3-Year Total | $374 | $142,720 | $179,884 |
Break-Even Analysis
Cloud vs API
Cloud GPU never breaks even - API cheaper
Self-Hosted vs Cloud
Breaks even in ~21 months
Recommendations
API services looks cheapest here
At 10M tokens/month it is the lowest three-year cost at this volume, with no infrastructure to run — about $374 over three years versus $142,720 for Cloud GPU.
High hardware requirements
Consider model quantization or smaller alternatives
How to read this comparison
The three options fail in different directions. A managed API costs nothing when idle and everything at scale, because you pay per token forever. A cloud GPU flips that: the meter runs on time, not tokens, so it rewards steady utilization and punishes bursty traffic — an instance at 5% utilization still bills for 100% of the hour. Self-hosting has the lowest marginal cost per token but the highest fixed cost and the longest payback, and it only wins when volume is both high and predictable.
The break-even figures are the part worth acting on. If your volume sits far below the crossover point, the decision is already made — use an API and spend the engineering time elsewhere. If you are near it, remember that the estimate assumes your GPU stays busy; halve the utilization and the crossover moves substantially. Before committing, confirm the memory footprint of your exact model and context length in the VRAM Calculator, check which card actually fits with Can I Run It, and read Open vs Closed Models for the governance side of the same decision.
Frequently asked questions
Is self-hosting an LLM cheaper than using an API?
Only above a break-even token volume. API cost scales linearly with usage and carries no infrastructure work, while self-hosting is mostly a fixed cost you pay whether the GPU is busy or idle. Below the break-even point, and for spiky traffic, a managed API is usually cheaper once engineering time is counted.
Why is the GPU not the biggest line item in self-hosting?
Because people are. A GPU is a one-off purchase amortized over about three years, but on-call, upgrades, and MLOps time recur every month. In this model, partial engineering headcount typically outweighs the hardware within the first year.
How accurate are these numbers?
They are planning estimates, not quotes. Per-token pricing varies widely by model size and by your input/output ratio, and cloud GPU rates differ several-fold between hyperscalers and specialist providers. Use the shape of the comparison and the break-even point, then confirm against current provider pricing before committing budget.
What drives the hardware tier I need?
The model's memory footprint. Weights in FP16 are roughly two bytes per parameter, plus the KV cache that grows with context length. Quantizing to 8-bit or 4-bit can drop you to a cheaper GPU tier, which is often the single largest cost lever available.