24 GB class · Runpod
L4
Within memory estimate · 14.54 GiB remaining
$0.59/ instance-hour · USD · one GPU
$59.00 for 100 hours
Available at observation ·
Inspect quote & conditions →Check provider pricing ↗Size your model. Plan your compute.
LLM VRAM Calculator
Find how much GPU memory your model needs. Set context and concurrency, check your existing card, then compare rental candidates and a compute budget.
Your inference workload
Single GPU · FP16 KV cacheEstimated memory · single GPU
7.81 GiB
8.38 GB · Q4_K_M
- Weight files
- 4.68 GiB
- FP16 KV cache
- 1.13 GiB
- Runtime reserve
- 2.00 GiB
GeForce RTX 4090 · 24 GB class
Within the memory budget.
14.54 GiB remaining · 22.35 GiB capacity budget.
At 1 concurrent sequence, memory allows up to 40,960 tokens per sequence (model configuration limit) under these assumptions.
Capacity estimate only. Check the exact GGUF file, engine, GPU backend and workload before deployment.
Inspect this GPU →Qwen3 8B · 8,192 total context tokens per sequence × 1 concurrent. Weights use reviewed file sizes.
Inspect the selected model’s evidence
Qwen3 8B · 36 layers · 8 KV heads · 128 head dimension · model configuration limit 40,960 tokens.
Reviewed GGUF files ↗ · Architecture configuration ↗.
Weight revision 7c41481f57cb95916b40956ab2f0b139b296d974; checked 8 Nov 2026. Repository license label: apache-2.0; read the model's terms before use.
- Q4_K_M: 5.03 GB / 4.68 GiB across 1 weight file.
- Q8_0: 8.71 GB / 8.11 GiB across 1 weight file.
From memory to a budget
GPU rental candidates for this configuration.
16 collected quotes pass the single-GPU memory and freshness checks. Showing the lowest collected rate in each of up to three memory classes, for 100 billable hours.
48 GB class · Runpod
RTX A6000
Within memory estimate · 36.90 GiB remaining
$0.59/ instance-hour · USD · one GPU
$59.00 for 100 hours
Available at observation ·
Inspect quote & conditions →Check provider pricing ↗32 GB class · Runpod
RTX 5090
Within memory estimate · 21.99 GiB remaining
$1.19/ instance-hour · USD · one GPU
$119.00 for 100 hours
Available at observation ·
Inspect quote & conditions →Check provider pricing ↗Runpod coverage only; this is not a market-wide cheapest-GPU claim. Quotes older than 48 hours, unknown rates and reported-unavailable configurations are excluded. Stock is dated, not reserved. Compute budgets exclude storage, transfers, tax and other charges. Include charged idle and startup time in your billable hours. No throughput or task-quality equivalence is implied.
Compare all 16 qualifying quotes
| GPU | Capacity class | USD / hour | 100 hours | Observed |
|---|---|---|---|---|
| L4 | 24 GB | $0.59 | $59.00 | 9 Nov 2026 |
| RTX A6000 | 48 GB | $0.59 | $59.00 | 9 Nov 2026 |
| RTX 4090 | 24 GB | $0.89 | $89.00 | 9 Nov 2026 |
| RTX 6000 Ada | 48 GB | $0.99 | $99.00 | 9 Nov 2026 |
| L40S | 48 GB | $1.09 | $109.00 | 9 Nov 2026 |
| RTX 5090 | 32 GB | $1.19 | $119.00 | 9 Nov 2026 |
| A100 PCIe | 80 GB | $1.19 | $119.00 | 8 Nov 2026 |
| A100 SXM | 80 GB | $1.39 | $139.00 | 8 Nov 2026 |
| A100 PCIe | 80 GB | $1.79 | $179.00 | 9 Nov 2026 |
| A100 SXM | 80 GB | $1.79 | $179.00 | 9 Nov 2026 |
| RTX PRO 6000 Blackwell Server Edition | 96 GB | $2.49 | $249.00 | 9 Nov 2026 |
| H200 SXM | 141 GB | $3.59 | $359.00 | 8 Nov 2026 |
| H100 SXM | 80 GB | $3.99 | $399.00 | 9 Nov 2026 |
| H200 SXM | 141 GB | $5.29 | $529.00 | 9 Nov 2026 |
| B200 | 180 GB | $7.99 | $799.00 | 9 Nov 2026 |
| B300 | 288 GB | $8.99 | $899.00 | 9 Nov 2026 |
Buying instead? 11 reviewed GPUs pass the capacity check
These are capacity candidates, not current retail offers or a performance ranking. Current seller price and stock are unverified. Check engine and backend support, board variant, power, cooling and host requirements.
GeForce RTX 5070NVIDIA · 12 GB classSpecifications & purchase evidence →GeForce RTX 5060 Ti 16GBNVIDIA · 16 GB classSpecifications & purchase evidence →Arc Pro B60Intel · 24 GB classSpecifications & purchase evidence →GeForce RTX 3090NVIDIA · 24 GB classSpecifications & purchase evidence →GeForce RTX 4090NVIDIA · 24 GB classSpecifications & purchase evidence →Radeon RX 7900 XTXAMD · 24 GB classSpecifications & purchase evidence →GeForce RTX 5090NVIDIA · 32 GB classSpecifications & purchase evidence →Radeon AI PRO R9700AMD · 32 GB classSpecifications & purchase evidence →RTX 6000 Ada GenerationNVIDIA · 48 GB classSpecifications & purchase evidence →RTX A6000NVIDIA · 48 GB classSpecifications & purchase evidence →RTX PRO 6000 Blackwell WorkstationNVIDIA · 96 GB classSpecifications & purchase evidence →
A starting point, with context included
LLM memory requirements by model.
Q4_K_M weight files + FP16 KV cache + 2 GiB runtime reserve, one concurrent sequence. These are capacity estimates for the reviewed files, not benchmarks. Change the settings above for your workload.
| Model | Weight files | 8K context | 32K context | 128K context |
|---|---|---|---|---|
| DeepSeek R1 Distill Qwen 7B | 4.36 GiB | 6.80 GiB | 8.11 GiB | 13.36 GiB |
| Qwen2.5 Coder 7B Instruct | 4.36 GiB | 6.80 GiB | 8.11 GiB | Outside model limit |
| Llama 3.1 8B Instruct | 4.58 GiB | 7.58 GiB | 10.58 GiB | 22.58 GiB |
| Qwen3 8B | 4.68 GiB | 7.81 GiB | 11.18 GiB | Outside model limit |
| Qwen3 14B | 8.38 GiB | 11.63 GiB | 15.38 GiB | Outside model limit |
| Qwen3 32B | 18.40 GiB | 22.40 GiB | 28.40 GiB | Outside model limit |
| DeepSeek R1 Distill Qwen 32B | 18.49 GiB | 22.49 GiB | 28.49 GiB | 52.49 GiB |
| Qwen2.5 Coder 32B Instruct | 18.49 GiB | 22.49 GiB | 28.49 GiB | Outside model limit |
| Llama 3.3 70B Instruct | 39.60 GiB | 44.10 GiB | 51.60 GiB | 81.60 GiB |
What goes into the number
How this LLM memory calculator works.
Weights + cache + reserve
Weight bytes are the total sizes of the reviewed GGUF files at a pinned repository revision. We count either the whole artifact or its complete shard set, never both.
KV bytes = 2 × layers × KV heads × head dimension × context tokens × concurrent sequences × 2 bytes
The first 2 accounts for keys and values; the last 2 is FP16 storage. The model weights are shared across sequences. The reserve covers an editable allowance for runtime, activations and buffers, not a measured peak.
Read the result as a capacity estimate
1 GB = 1,000,000,000 bytes; 1 GiB = 1,073,741,824 bytes. Manufacturer capacity labels use a conservative decimal-GB budget here. Actual free device memory and engine allocations can differ.
Preset context limits come from the pinned configuration without assuming RoPE extensions. FP16 KV, full context, no prefix sharing, all weights on one GPU. Only the reviewed dense architectures and formats are included. Different formats and quantization levels do not establish equivalent quality.
Hugging Face memory-estimation guidance ↗ · llama.cpp GPU backend setup ↗.
All reviewed model sources
DeepSeek R1 Distill Qwen 7B · 28 layers · 4 KV heads · 128 head dimension · model configuration limit 131,072 tokens.
Reviewed GGUF files ↗ · Architecture configuration ↗.
Weight revision 361004151d4f4f6b446dc5e6d46fbf4422a80d5f; checked 8 Nov 2026.
- Q4_K_M: 4.68 GB / 4.36 GiB across 1 weight file.
- Q8_0: 8.10 GB / 7.54 GiB across 1 weight file.
Qwen2.5 Coder 7B Instruct · 28 layers · 4 KV heads · 128 head dimension · model configuration limit 32,768 tokens.
Reviewed GGUF files ↗ · Architecture configuration ↗.
Weight revision 13fb94bfda8c8cf22497dc57b78f391a9acb426a; checked 8 Nov 2026. Repository license label: apache-2.0; read the model's terms before use.
- Q4_K_M: 4.68 GB / 4.36 GiB across 1 weight file.
- Q8_0: 8.10 GB / 7.54 GiB across 1 weight file.
Llama 3.1 8B Instruct · 32 layers · 8 KV heads · 128 head dimension · model configuration limit 131,072 tokens.
Reviewed GGUF files ↗ · Architecture configuration ↗ (public mirror).
Weight revision bf5b95e96dac0462e2a09145ec66cae9a3f12067; checked 8 Nov 2026. Repository license label: llama3.1; read the model's terms before use.
- Q4_K_M: 4.92 GB / 4.58 GiB across 1 weight file.
- Q8_0: 8.54 GB / 7.95 GiB across 1 weight file.
Qwen3 8B · 36 layers · 8 KV heads · 128 head dimension · model configuration limit 40,960 tokens.
Reviewed GGUF files ↗ · Architecture configuration ↗.
Weight revision 7c41481f57cb95916b40956ab2f0b139b296d974; checked 8 Nov 2026. Repository license label: apache-2.0; read the model's terms before use.
- Q4_K_M: 5.03 GB / 4.68 GiB across 1 weight file.
- Q8_0: 8.71 GB / 8.11 GiB across 1 weight file.
Qwen3 14B · 40 layers · 8 KV heads · 128 head dimension · model configuration limit 40,960 tokens.
Reviewed GGUF files ↗ · Architecture configuration ↗.
Weight revision 530227a7d994db8eca5ab5ced2fb692b614357fd; checked 8 Nov 2026. Repository license label: apache-2.0; read the model's terms before use.
- Q4_K_M: 9.00 GB / 8.38 GiB across 1 weight file.
- Q8_0: 15.70 GB / 14.62 GiB across 1 weight file.
Qwen3 32B · 64 layers · 8 KV heads · 128 head dimension · model configuration limit 40,960 tokens.
Reviewed GGUF files ↗ · Architecture configuration ↗.
Weight revision 938a7432affaec9157f883a87164e2646ae17555; checked 8 Nov 2026. Repository license label: apache-2.0; read the model's terms before use.
- Q4_K_M: 19.76 GB / 18.40 GiB across 1 weight file.
- Q8_0: 34.82 GB / 32.43 GiB across 1 weight file.
DeepSeek R1 Distill Qwen 32B · 64 layers · 8 KV heads · 128 head dimension · model configuration limit 131,072 tokens.
Reviewed GGUF files ↗ · Architecture configuration ↗.
Weight revision 1dc8cf9ffa5dd333057ea1b09ccf4772d8726dec; checked 8 Nov 2026.
- Q4_K_M: 19.85 GB / 18.49 GiB across 1 weight file.
- Q8_0: 34.82 GB / 32.43 GiB across 1 weight file.
Qwen2.5 Coder 32B Instruct · 64 layers · 8 KV heads · 128 head dimension · model configuration limit 32,768 tokens.
Reviewed GGUF files ↗ · Architecture configuration ↗.
Weight revision 9d3053fce650fe1cdbdb75998c2a87add9d178ef; checked 8 Nov 2026. Repository license label: apache-2.0; read the model's terms before use.
- Q4_K_M: 19.85 GB / 18.49 GiB across 1 weight file.
- Q8_0: 34.82 GB / 32.43 GiB across 1 weight file.
Llama 3.3 70B Instruct · 80 layers · 8 KV heads · 128 head dimension · model configuration limit 131,072 tokens.
Reviewed GGUF files ↗ · Architecture configuration ↗ (public mirror).
Weight revision b6c5c9f176f3279204034e1d16d393105e95cb88; checked 8 Nov 2026.
- Q4_K_M: 42.52 GB / 39.60 GiB across 1 weight file.
- Q8_0: 74.98 GB / 69.83 GiB across 2 weight files.
- F16: 141.12 GB / 131.43 GiB across 4 weight files.
LLM VRAM planning questions
How much VRAM do I need to run an LLM?
Add the model weights, a KV cache for your context and concurrent requests, and room for the inference engine. The model’s parameter count alone is not enough. This calculator uses actual reviewed GGUF file sizes and full-attention architecture data, then adds an editable runtime reserve. It estimates fully GPU-resident inference, not training.
Can an RTX 4090 or RTX 5090 run my model?
Use the GPU selector above: this tool assigns conservative 24 GB and 32 GB capacity budgets to those cards. The reviewed Llama 3.3 70B Q4_K_M files alone exceed both budgets. Smaller models may fit, depending on format, context and concurrency. A capacity result does not establish engine compatibility or an acceptable response speed.
Is 4-bit memory simply parameters × 0.5 bytes?
That is an idealized weight-storage calculation. Real GGUF quantizations include metadata and can keep some tensors at higher precision. Q4_K_M is not the same as every other 4-bit format. We use the combined bytes of the exact reviewed files, and calculate KV cache and runtime reserve separately.
Does concurrency mean the number of people in my company?
No. It means sequences being processed at the same time, with a full cache for each sequence in this scenario. Ten employees do not necessarily create ten simultaneous requests. Weights are shared; the estimated cache multiplies with concurrency. Prefix sharing, dynamic batching and cache allocation policies can change actual usage.
Why can this differ from Ollama, llama.cpp or vLLM?
Engines use different allocation policies, buffers, padding, batching and GPU backends. The reviewed artifacts here are GGUF; another engine or format can need a different weight budget. The 2 GiB default reserve is a planning allowance, not a measurement. Compare your engine logs and actual free VRAM, and adjust the allowance.
Can I use this for training, MoE models or multiple GPUs?
This first version covers dense, full-attention models on one GPU. Training adds gradients, optimizer state and other allocations. MoE weights, hybrid or sliding attention, recurrent state and MLA need architecture-specific handling. Multiple cards require an explicit split and per-device calculation; their memory is not automatically pooled.
How current are the GPU prices?
The shortlist uses dated Runpod observations from our scheduled price collection. Only known USD single-GPU instance-hour rates, reported available at observation and no more than 48 hours old at the comparison time, qualify. This is one provider’s collected coverage, not a live stock reservation or a whole-market comparison.
Choose capacity around the work you need to do.
Memory is the first check. Your latency, task quality, utilization and operating costs determine the next step.
Buy, rent or use an API? →
Compare workload budgets, including utilization and operating costs.
Inspect GPU specifications →
Review exact variants, published test evidence and purchase references.
Pricing correction or sourcing discussion →
Share the model, configuration and listing you want us to review.