AI Model Wiki

    Skip to main content

    Local LLM Hardware Calculator

    Estimate the VRAM and GPU class you need to run a local LLM. Numbers include weights, KV cache, and activation overhead.

    Your model

    Common sizes: 7, 13, 30, 70, 405, 671 (DeepSeek-V3).

    Common consumer choice; modest quality drop

    VRAM breakdown

    Weights
    42.0 GB
    KV cache @ 8,192 tok
    14.0 GB
    Activations + overhead
    5.6 GB
    Recommended VRAM (+10% buffer)
    68 GB

    Estimates only. Real KV cache size depends on attention type, layer count, and grouped-query attention; long-context models (sliding window, Mamba) can be much lighter.

    Hardware that fits

    • Apple M3 Ultra 192GB
      Up to ~400B Q4 (DeepSeek-V3 class)
      160 GB
      57% headroom
    • A100 80GB / H100 80GB
      70B FP16; 405B Q4
      75 GB
      9% headroom
    • H100 NVL / B200 192GB
      405B FP16; 1T Q4 (multi-GPU)
      180 GB
      62% headroom

    Speed estimate

    On the smallest GPU that fits (Apple M3 Ultra 192GB), this configuration is likely to run around 26 tokens/sec. Your target was 15 tok/s. You should hit it.

    Throughput rule-of-thumb extrapolated from Q4 70B on RTX 4090 ≈ 20 tok/s. Multi-GPU, NVLink, and software stack (llama.cpp vs vLLM vs TensorRT-LLM) materially change the result. See the model page for benchmarked numbers where available.