Local LLM Hardware Calculator
Estimate the VRAM and GPU class you need to run a local LLM. Numbers include weights, KV cache, and activation overhead.
Your model
Common sizes: 7, 13, 30, 70, 405, 671 (DeepSeek-V3).
Common consumer choice; modest quality drop
VRAM breakdown
Estimates only. Real KV cache size depends on attention type, layer count, and grouped-query attention; long-context models (sliding window, Mamba) can be much lighter.
Hardware that fits
- Apple M3 Ultra 192GBUp to ~400B Q4 (DeepSeek-V3 class)160 GB57% headroom
- A100 80GB / H100 80GB70B FP16; 405B Q475 GB9% headroom
- H100 NVL / B200 192GB405B FP16; 1T Q4 (multi-GPU)180 GB62% headroom
Speed estimate
On the smallest GPU that fits (Apple M3 Ultra 192GB), this configuration is likely to run around 26 tokens/sec. Your target was 15 tok/s. You should hit it.
Throughput rule-of-thumb extrapolated from Q4 70B on RTX 4090 ≈ 20 tok/s. Multi-GPU, NVLink, and software stack (llama.cpp vs vLLM vs TensorRT-LLM) materially change the result. See the model page for benchmarked numbers where available.