Local AI Model Memory Calculator (RAM and VRAM for GGUF Models)
A local model has to fit in memory to run at a usable speed: video memory (VRAM) on a dedicated GPU, or the shared pool on computers with unified memory. Two things take most of the space, NVIDIA's inference guide explains: the model weights and the KV cache, which grows with the context length.
This gives a floor, not an exact figure. Apps also need working memory for compute buffers, and "_M" or "_S" files mix several quantization types. If you know the GGUF file's size, enter it: that's the best weight figure. After loading,
ollama psshows what was actually used.
This calculator needs JavaScript turned on. It runs entirely in your browser: nothing you type is sent anywhere or stored, and the page sets no cookies.
Estimated memory
| Model weights | — |
|---|---|
| KV cache | — |
| Your margin | — |
| Total (weights + KV cache + margin) | — |
Formulas and assumptions
- Weights (bytes) = parameters × bits per weight ÷ 8. Bits per weight for Q6_K, Q5_K, Q4_K, Q3_K, Q2_K, IQ4_XS and IQ3_S are from Hugging Face's GGUF quantization table. Q8_0 and Q4_0 store blocks of 32 weights with a 16-bit scale (34 and 18 bytes per block in llama.cpp's
ggml-common.h), which works out to 8.5 and 4.5 bits. - KV cache (bytes) = 2 × layers × (KV heads × head dimension) × context tokens × bytes per value × parallel requests. This is NVIDIA's per-token formula with the number of key/value heads, since models using grouped-query attention store fewer KV heads than attention heads. f16 uses 2 bytes per value; q8_0 and q4_0 use 1.0625 and 0.5625 (the same block layouts), matching the Ollama FAQ's "about ½" and "about ¼".
- Parallel requests multiply the context: the Ollama FAQ says a 2K context with 4 parallel requests becomes an 8K context.
- Units: GB = 10⁹ bytes; GiB = 2³⁰ bytes. Ollama's context defaults are set by GiB of VRAM.
- Not included: compute buffers, the app's own memory, your operating system and other programs, and vision or embedding add-ons. We don't add a fixed overhead because none of the sources publishes one; add your own margin if you want headroom.
Worked examples
From NVIDIA's guide: Llama 2 7B (32 layers, 4,096 hidden size, no grouped-query attention) at 4,096 tokens in 16-bit precision needs 1 × 4,096 × 2 × 32 × 4,096 × 2 bytes = 2,147,483,648 bytes, about 2 GB, of KV cache.
Qwen3-8B at Q4_K, 4,096 tokens: weights 8.2 billion × 4.5 ÷ 8 = 4.61 GB. KV cache 2 × 36 × (8 × 128) × 4,096 × 2 bytes = 0.60 GB. Floor: about 5.2 GB, before buffers and the rest of your system. At 32,768 tokens the KV cache alone is 4.83 GB in f16, or about 2.57 GB as q8_0.
Sources
- Hugging Face Docs: GGUF (quantization types and bits per weight)
- llama.cpp: ggml-common.h (Q8_0 and Q4_0 block layouts)
- NVIDIA Technical Blog: Mastering LLM Techniques: Inference Optimization (KV cache formula)
- Ollama documentation: FAQ (KV cache quantization, parallel requests, ollama ps)
- Ollama documentation: Context length
- Qwen3-8B model card and config.json
- Qwen3-14B model card and config.json
- Qwen3-32B model card and config.json
- Qwen2.5-7B-Instruct model card and config.json
Sources checked October 29, 2026.