Can I run a local LLM with 8 GB of VRAM?
Yes — 7-8B models at q4 quants (about 5 GB on disk) run fully in 8 GB of VRAM with room for context. Bigger Mixture-of-Experts models can also run by offloading idle experts to system RAM.
How much VRAM do I need to run a 70B model?
Llama-3.3-70B-class dense models need roughly 38-43 GB at q4-class quants, so a 48 GB card runs them fully in VRAM. With less VRAM they need partial CPU offload, which is noticeably slower for dense models.
Does system RAM matter for local LLMs?
For Mixture-of-Experts models, a lot: only a few billion parameters are active per token, so the idle experts can sit in system RAM (llama.cpp's --n-cpu-moe) while the GPU holds the active path — that is how a 120B-class MoE runs on a 24 GB card.
Are these fit results exact?
They're honest estimates from a Hugging-Face-verified catalog: weights + KV cache (computed per attention type — full, sliding-window, or hybrid-linear, which most current Qwen models use) + a compute buffer, with big MoE models counted as runnable via expert-offload to system RAM. Results are ranked most-capable-first. TurboLLM re-checks against your exact hardware before you load and then measures actual tokens/sec — the estimates here pick the shortlist, your hardware settles it.