Best local LLMs for the RTX 5080 (16 GB) — updated July 2026
The RTX 5080 is the fastest 16 GB card you can buy — but for local LLMs the binding constraint
is VRAM, not compute, so it fits exactly the same models as the cheaper
RTX 5070 Ti. Fully in VRAM, that ceiling is a
27B dense at IQ4_XS; the 35B-A3B MoE's Q3_K_M (16.6 GB)
just spills past 16 GB — auto-fit nudges a few experts to system RAM — and with expert
offload, a huge coding MoE like Qwen3-Coder-Next runs with its experts in
system RAM while attention and KV cache stay on the GPU. What the extra money buys is speed:
~7% more memory bandwidth (960 vs 896 GB/s) and roughly 20% more compute, which shows most
in prompt processing. If you want bigger fits rather than faster ones, the step up
is the RTX 5090's 32 GB.
Numbers you can trust
TurboLLM measures real tokens/sec on your card and shows a
VRAM-fit verdict before you load — no guessed speeds in the app, ever.
Every speed on this page is labeled with its source: the only numbers we measured ourselves
came from an RTX 5070 Ti 16 GB — the 5080's sibling card with the same VRAM — and we say
so wherever they appear. We found no broadly published community benchmarks for the 5080
on current-generation models, so we don't print any.
The shortlist below is the models hub's 16 GB tier with the fit worked out for this card, plus the 8 GB tier's all-rounder promoted to a near-lossless quant. The fits are identical to the RTX 5070 Ti's — same 16 GB — the 5080 just runs them faster.
Qwen3.6 35B-A3B
by Alibaba · GGUF
MoE · ~3B active
toolsthinkingcode
The daily driver for 16 GB — only ~3B params active per token, so it's dramatically faster than a dense model of similar quality. This is the model behind our RTX 5070 Ti measurements, and the 5080 shares that card's VRAM budget exactly.
RTX 5080:Q3_K_M (16.6 GB) just spills past 16 GB — auto-fit nudges a few experts to system RAM · or Q4_K_M (22 GB) + expert offload · 256K context
The dense sibling — a touch slower than the MoE but rock-solid quality, and it fits fully in VRAM at a mid quant. Dense generation is where the 5080's extra bandwidth over the 5070 Ti earns its keep.
RTX 5080:IQ4_XS (15.4 GB) or Q3_K_M (13.6 GB) in VRAM · Q4_K_M (16.8 GB) + light offload · 256K context
The newest agentic coding MoE. Its experts live in system RAM while attention and KV stay on the GPU, so it runs on a 16 GB card — TurboLLM's auto-fit sets the split for you.
RTX 5080:IQ2–IQ3 with heavy CPU-expert offload (needs system RAM) · even better on 24 GB+ · large context
The vision pick — current-gen vision-language model with strong OCR and spatial reasoning. Its tiny footprint leaves most of the 16 GB free for images and context.
RTX 5080:Q4_K_M (5.0 GB) with plenty of headroom for images + context · 256K context
The 8 GB tier's all-rounder, promoted: on 16 GB it runs near-lossless at Q8_0 with a toggleable thinking mode — and a small dense model on the fastest 16 GB card is about as snappy as local inference gets.
RTX 5080:Q8_0 (8.7 GB) fully in VRAM with generous context headroom · 128K context
We haven't benchmarked an RTX 5080, and we found no broadly published community numbers for it
on the current-generation picks above — so no table of made-up figures. What we do have
is unusually relevant: our own measurements from the RTX 5070 Ti 16 GB, the
5080's sibling — same Blackwell generation, same 16 GB, about 7% less memory bandwidth
(896 vs 960 GB/s). On that card, Qwen3.6-35B-A3B at a full 200K context ran at
74.7 t/s (q4_0 KV cache) and 72.3 t/s (q8_0 KV) in TurboLLM —
versus 61.0 and ~66 t/s for the same setups in LM Studio. Qwen3.6-27B dense at 200K
context ran at 24.6 t/s with TurboLLM's turbo4 KV quantization versus
11.4 t/s in LM Studio at q8_0 KV (2.2×), with prefill at 1,288 vs 757 tok/s. All of those are
our measurements on the 5070 Ti — not the 5080.
Because token generation is memory-bandwidth-bound, the 5080's extra bandwidth should put it at
or slightly above those figures on the same models — but that's reasoning, not a measurement,
and prompt processing (where its ~20% compute advantage matters more) is harder to extrapolate.
TurboLLM settles it empirically: load a model and it benchmarks real generation on your 5080,
storing the measured tokens/sec in the model list. Weighing this card against its neighbors:
the RTX 5070 Ti runs the identical shortlist a step slower for
less money, and the RTX 5090 is the card that changes what
fits, not just how fast it runs.
How TurboLLM auto-fits the RTX 5080
Load any model and TurboLLM benchmarks it on your 5080, storing the measured
tokens/sec in the model list — live numbers from actual generation, not estimates.
Before you load, every quant gets a VRAM-fit verdict against your real 16 GB,
including KV-cache growth at your chosen context — the difference between IQ4_XS fitting the
27B and Q4_K_M needing offload is exactly the kind of margin it catches. Auto-fit then picks
the GPU-layer count and the MoE expert-offload split for you — it's what puts
Qwen3-Coder-Next's experts in system RAM automatically. Details in
the auto-tune docs.
FAQ
Can the RTX 5080 run a 70B model?
Not usably. A dense 70B is about 43 GB at q4_K_M, so on 16 GB most of the model
would spill to system RAM and generation drops to a crawl. The better big-model path on this
card is a Mixture-of-Experts: Qwen3.6 35B-A3B's Q3_K_M (16.6 GB)
just spills past 16 GB — auto-fit nudges a few experts to system RAM — and
Qwen3-Coder-Next runs with its experts offloaded to system RAM while attention
and the KV cache stay on the GPU.
Is the RTX 5080 better than the RTX 5070 Ti for local LLMs?
For what fits, no — both have 16 GB, so they run exactly the same models at exactly the same
quants. For speed, yes: the 5080 has about 7% more memory bandwidth (960 vs 896 GB/s) and
roughly 20% more compute, so the same models generate somewhat faster and prompt processing is
quicker. If you want bigger models rather than faster ones, the jump that matters is the
RTX 5090's 32 GB.
What quant should I use on 16 GB of VRAM?
For a 27B dense model, IQ4_XS (15.4 GB) just fits fully in VRAM and
Q3_K_M (13.6 GB) leaves more context headroom. For the 35B-A3B MoE,
Q3_K_M (16.6 GB) just spills past 16 GB — auto-fit nudges a few experts to
system RAM — and Q4_K_M runs with expert offload. Models of 8B and under
run near-lossless at Q8_0. TurboLLM shows a VRAM-fit verdict for each quant before
you download — the naming is decoded in Quantization explained.
How fast will Qwen3.6 35B-A3B run on the RTX 5080?
We haven't measured the 5080, so we won't print a number for it. On its sibling card — an
RTX 5070 Ti with the same 16 GB and about 7% less memory bandwidth — we
measured 74.7 tokens/sec at a 200K context in TurboLLM. Generation is
memory-bandwidth-bound, so the 5080 should land in the same neighborhood or slightly above,
and TurboLLM measures the exact figure on your card the moment you load the model.
Do I need to compile anything to run local LLMs on the RTX 5080?
No. TurboLLM detects the card and downloads a prebuilt CUDA engine with Blackwell support
automatically — llama.cpp or a community fork — with nothing to compile. Running
npx turbollm is the whole setup.
Try it on your 5080
$npx turbollm
One command: it detects the 5080, downloads a CUDA engine, and opens the UI. New here? Start
with Install & first run and
Quantization explained. On different hardware, see the
RTX 5070 Ti (same fits, lower price) or the
RTX 5090 (32 GB — bigger fits) guides.