VRAM for an LLM is the amount of memory on your graphics card that a model and its working data have to fit into in order to run. The calculator below estimates that figure from three inputs: the model size in billions of parameters, the quantization you plan to use, and how long a context you want to feed it. Enter those numbers and it returns an approximate VRAM target so you can tell whether a model fits the card you have.
What VRAM Is and Why Local LLMs Need It
VRAM is the dedicated memory built into a graphics card. When you run a language model locally, every weight in the model has to be loaded into that memory so the GPU can read it during each step of generation. On top of the weights, the runtime keeps a working area called the KV cache that grows with the length of the conversation, plus a fixed amount of overhead for the program itself. If the total exceeds the card’s VRAM, the model either fails to load or spills into system memory and slows to a crawl. Knowing the VRAM figure before you download a model tells you in advance whether it will run on your hardware.
How to Use It
- Enter the model size in billions of parameters. A model labeled 7B is 7, a 13B model is 13, and so on.
- Pick the quantization you plan to run. FP16 is the full size, 8-bit halves it, and 4-bit quarters it.
- Set the context length in tokens. This is the number of tokens the model can hold at once, often 4096 or 8192.
- Read the recommended VRAM figure and compare it to your card. The result updates as you change any input.
- Round up to the next real card size, since a model that just barely fits leaves no room for the operating system or other apps.
How the Estimate Works
The largest part of the figure is the weights, and the math behind it is direct. Each parameter occupies a fixed number of bytes that depends on the quantization, so the weight size is the parameter count multiplied by the bytes per parameter, then converted into gigabytes.
What Quantization Does
Quantization stores each weight using fewer bits. FP16 keeps the full sixteen-bit precision, 8-bit and 4-bit compress each weight into less space. Lower precision shrinks the memory the model needs, which is what lets larger models run on smaller cards. The tradeoff is a small loss of accuracy that grows as the precision drops, though 8-bit and good 4-bit formats keep most models usable.
| Quantization | Bytes per parameter | Weight size of a 7B model | Memory tradeoff |
|---|---|---|---|
| FP16 (16-bit) | 2 | ~13 GB | Full precision, largest memory use |
| 8-bit | 1 | ~6.5 GB | Half the memory, minor quality loss |
| 4-bit | 0.5 | ~3.3 GB | Quarter the memory, more quality loss |
Choosing a GPU
Match the recommended VRAM figure to a card that has at least that much memory, then leave headroom. Common consumer tiers are 8 GB, 12 GB, 16 GB, and 24 GB; data-center cards reach 48 GB and 80 GB. A 7B model at 4-bit fits comfortably on a 12 GB card, while a 13B model in FP16 pushes past 24 GB. If a model is close to a card’s limit, drop to a lower quantization or pick a smaller model rather than running with no spare memory.
Read the Result as an Approximation
This calculator gives an estimate, not an exact requirement. Real VRAM use depends on the runtime you choose, the batch size, how the KV cache is stored, and how long your context actually grows during use. Different inference engines pack memory differently, and some reserve extra for speed. Treat the figure as a planning number that tells you which card class to aim for, and confirm against the documentation of the specific model and runtime you settle on.
When to Use It
Reach for this tool before you download a model, before you buy a graphics card, and whenever you are deciding which quantization to run. It answers the practical question of whether a given model will fit a given card, and it shows how much memory you save by stepping down from FP16 to 8-bit or 4-bit. That makes it a quick first check when planning a local setup.
Last Thoughts on Estimating VRAM for Local LLMs
The memory a local model needs comes down to a simple chain: count the parameters, multiply by the bytes each one takes under your chosen quantization, add a margin for the runtime and context, and round up to a real card. That gives you a reliable target without trial and error, and it makes the cost of precision visible, since dropping to 4-bit can move a model from a card you would have to buy to one you already own.
Run your model size through the calculator, then plan the rest of the build with our other free online tools. To estimate what running a model costs to use, see the LLM cost calculator, and to understand how context length is counted, try the words to tokens converter.
Key Takeaways:
- VRAM need is dominated by the model weights, which equal the parameter count times the bytes per parameter.
- FP16 uses 2 bytes per parameter, 8-bit uses 1, and 4-bit uses 0.5, so quantization can cut memory use by half or three quarters.
- Add roughly twenty percent over the weights for the runtime and KV cache, plus a small amount that grows with context length.
- Round the result up to the next real card size so the operating system and other programs still have room.
- The figure is an approximation; real use varies by runtime, batch size, and context.
- Use it to check whether a model fits your card and to compare the memory savings of each quantization level.
Frequently Asked Questions (FAQs)
How much VRAM do I need to run a 7B model?
In FP16 a 7B model holds about 13 GB of weights, so with overhead it wants roughly 16 GB of VRAM. At 4-bit the same model drops to about 3.3 GB of weights and fits comfortably on a 12 GB card. The exact figure depends on context length and runtime, which is why the calculator adds a margin and rounds up.
How do I calculate VRAM from a model’s parameter count?
Multiply the parameter count by the bytes per parameter set by your quantization, then convert to gigabytes. FP16 is 2 bytes, 8-bit is 1 byte, and 4-bit is 0.5 bytes per parameter. Add about twenty percent for the runtime and KV cache to get a working VRAM target.
Does context length change how much VRAM I need?
Yes. Longer context grows the KV cache, the working memory the model uses to track the tokens it has seen. The weights stay fixed, but a long context adds memory on top of them. This tool includes a small term that scales with context length so the estimate reflects that.
Does 4-bit quantization really cut VRAM in half?
It cuts the weight memory to a quarter of FP16, since each parameter drops from 2 bytes to 0.5 bytes. That is the single biggest lever for fitting a larger model on a smaller card. The tradeoff is a modest loss of accuracy, which most models tolerate well at 4-bit.
Is this VRAM estimate exact?
No, it is an approximation meant for planning. Real VRAM use depends on the inference runtime, the batch size, how the KV cache is stored, and the actual context you run. Use the figure to pick a card class, then confirm against the documentation for your specific model and runtime.
Can I run an LLM without a GPU?
You can run smaller models on system RAM and a CPU, but generation is much slower than on a GPU. Quantized models in the 7B range are the most practical for CPU-only setups. A GPU with enough VRAM remains the fastest way to run a local model, which is what this calculator helps you size.
