GPU VRAM Estimator
Estimate how much GPU memory a model needs for inference or training, based on parameter count and precision.
How the estimate is calculated
Inference: VRAM ≈ params × bytes_per_param × 1.2, weights plus a rough activation/KV-cache buffer. Training with Adam or AdamW: VRAM ≈ params × 16 bytes, before activations. With fp16/bf16 mixed precision that is 2 bytes for the half-precision weights, 2 for the half-precision gradients, 4 for the fp32 master copy of the weights and 8 for Adam's two moment estimates (m and v) in fp32. With plain fp32 training it is 4 for the weights, 4 for the gradients and 8 for Adam, with no separate master copy. Some references, including Hugging Face's GPU memory guide, count gradients in fp32 during mixed precision, which gives 18 bytes per parameter. Activations come on top and depend on batch size and sequence length, so this mode does not include them. int8 and int4 are inference precisions here; the tool does not estimate full training at those.
Worked example (7B parameters)
| Scenario | Estimated VRAM |
|---|---|
| Inference, fp16 | 15.65 GiB |
| Inference, int4 | 3.91 GiB |
| Training, fp16/bf16 mixed precision (Adam), before activations | 104.31 GiB |
| Training, full fp32 (Adam), before activations | 104.31 GiB |
A 7B model fits comfortably on a single 24GB consumer card for fp16 inference, and int4 leaves enough headroom for a decent context window on top. Full fine-tuning with Adam needs around 104 GiB before activations, past what one high-end consumer GPU offers. LoRA and similar methods that update a small slice of parameters instead of all of them exist largely to dodge that number.
What this leaves out
Batch size and sequence length both push activation memory up in ways the flat 1.2x buffer does not capture, especially at long context where the KV cache can dominate. Gradient checkpointing, optimizer choice beyond Adam, and framework-specific fragmentation all shift the training number too. Size hardware from this, but confirm the actual footprint with a real run before signing a GPU rental invoice.
How to estimate VRAM in Python
Rule-of-thumb math, not a spec sheet. Your actual mileage depends on the framework, batch size, and sequence length you run with.