Will This LLM Fit on Your GPU

Pick a model, a quantization and a card, and get a straight answer with the arithmetic shown. Weight size uses measured bits per weight taken from 66 real GGUF files, not the nominal 4-bit figure most calculators assume, and the KV cache comes from each model's own published config rather than a rule of thumb.

Why the nominal bit count is the wrong number

A quantized checkpoint is not uniformly quantized. In the GGUF K-quant formats the attention and feed-forward weights carry the named precision, but embeddings, output projections and some per-layer tensors are kept wider, and every block stores scale and minimum values alongside its packed weights. The name tells you the dominant block type. It does not tell you the file size.

Measuring 66 published GGUF files across 11 models gives the real figure. Each file size below came from the x-linked-size header on the Hugging Face download URL, and each parameter count from the corresponding repository's safetensors index, both fetched on 22 August 2026.

Q4_K_M, the most downloaded format on Hugging Face, measures 4.914 bits per weight rather than 4. Q3_K_M is worse in relative terms at 4.003 against a nominal 3, a 33 percent overshoot, because the tensors held at higher precision are a larger share of a smaller total. Size a 32B model at a nominal 4 bits and you predict 16.0 GiB of weights. The real Q4_K_M file is 18.49 GiB. On a 24 GB card that gap is most of your context budget.

Applying the mean back to each model reproduces its real file size closely: the predicted weight size lands within 2 percent of the measured file for 10 of the 11 models, and the largest error in the set is 3.8 percent on the smallest model, where the fixed-precision tensors weigh most. That is the accuracy you should expect from the weights term here. The KV cache term is exact arithmetic on published config values, and the overhead term is your own allowance.

How the calculator works

Three terms are added. Two are computed, one is an allowance you control.

Parameter counts come from each repository's own safetensors metadata, so they are the counts the checkpoint actually stores rather than the rounded name. Qwen2.5 7B stores 7,615,616,512 parameters, not 7,000,000,000.

The leading 2 covers the key and the value tensor. The term that matters most here is kv_heads, not the attention head count: a grouped-query model with 8 KV heads behind 32 attention heads uses a quarter of the cache a multi-head model of the same width would. Every value is read from the model's published config, which is why the answer changes so much between architectures that look similar on paper. For a deeper treatment of this term on its own, see the KV cache calculator.

The third term is runtime overhead: the CUDA context, the compute buffers the framework allocates for the forward pass, and allocator fragmentation. It depends on your runtime, your driver and your batching, so this page does not pretend to measure it. It is a field you set, defaulting to 1 GiB, and the verdict tells you how much slack you have so you can judge whether that default is generous enough for your setup.

The verdict treats anything above 95 percent of card capacity as too tight to trust, because a card at 99 percent occupancy fails the moment a prompt runs long or the allocator fragments.

GPU capacities used

Every capacity below was confirmed against the manufacturer's own specification page on 22 August 2026. Two commonly requested cards, the RTX A6000 and the V100, are absent because their stated capacity could not be confirmed from a fetchable primary source, and a number that cannot be checked does not belong in a calculator that people size hardware with.

Capacity is the physical figure. Your display output, desktop compositor and any other process take a share before your model loads, so treat the fit verdict as an upper bound on a card you are also using to drive a monitor.

What this calculator does not model

It sizes single-GPU inference. It does not model training, which adds gradients and optimizer state and is usually several times the weight size, and for that the GPU memory calculator for training is the right page. It does not model tensor parallelism across several cards, CPU offload, or the paged attention schemes that let some servers exceed these figures by trading latency. Speculative decoding and draft models add a second set of weights this page does not count.