githot.
← All tools

TOOLS · AI & LLM

LLM VRAM Calculator

FreeQuick estimate

Saw a video about running an AI model locally and wonder if your machine can handle it? Pick a model, quantization and context to see how much memory it needs.

01 · Model
02 · Quantization

Bits per weight. Q4_K_M is the popular choice: small, with quality close to the original.

03 · Context

Longer context means a bigger KV cache. This model supports up to 131,072 tokens.

KV cache type

Q8_0 or Q4_0 shrink the context memory (llama.cpp: --cache-type-k/v; Ollama: OLLAMA_KV_CACHE_TYPE). Each request running at the same time needs its own KV cache.

Memory needed

About 6.2 GB

Model weights
4.5 GB
KV cache
1.0 GB
Buffers & runtime
0.7 GB

The model file is about 4.5 GB to download.

Which machines can run it?

  • RTX 3050 / 4060~7.5 GB usableRuns well
  • RTX 3060 12 GB~11.3 GB usableRuns well
  • RTX 4060 Ti 16 GB / 4080~15.0 GB usableRuns well
  • RTX 3090 / 4090~23 GB usableRuns well
  • RTX 5090~30 GB usableRuns well
  • Mac 16 GB~10.7 GB usableRuns well
  • Mac 24 GB~16.1 GB usableRuns well
  • Mac 36 GB~24 GB usableRuns well
  • Mac 64 GB~48 GB usableRuns well
  • Mac 128 GB~96 GB usableRuns well
  • H100 80 GB~75 GB usableRuns well

Estimated as weights + KV cache + buffers; real use varies by 10–20% between runtimes (llama.cpp, Ollama, LM Studio, vLLM). Macs share memory with the system.

This tool is free for now. If it helped, buy me a coffee ☕