Best local LLMs for the RTX 5090 (32 GB) — updated July 2026
The RTX 5090's 32 GB is the consumer VRAM ceiling. For dense models that ceiling is roughly the
50B class at q4 fully in VRAM — a dense 70B at q4_K_M (~43 GB) still
doesn't fit, so in today's open-weight lineup the practical win is a 27B–35B at q5–q6 with
huge KV-cache headroom. For MoE, expert offload runs a 120B-class model
like gpt-oss-120B with a lighter offload than any 24 GB card needs. Every fit on the
RTX 4090 page works here one quant step higher — and if you're on the
16 GB RTX 5080, that's a different set of fits entirely.
Numbers you can trust
TurboLLM measures real tokens/sec on your card and shows a
VRAM-fit verdict before you load — no guessed speeds in the app, ever.
Every speed on this page is labeled with its source: the only numbers we measured ourselves
came from a different card (an RTX 5070 Ti 16 GB), and everything quoted for the 5090 is
community-reported.
The shortlist below blends the models hub's tiers: its 24 GB+ picks bumped to the higher quants 32 GB allows, the 16 GB tier's dense Qwen3.6 27B promoted as the deep-context pick, plus the most reachable pick from its 48 GB tier — GLM-4.5-Air. Everything a 24 GB card runs also runs here at a higher quant, a longer context, or a lighter offload.
Qwen3.6 35B-A3B
by Alibaba · GGUF
MoE · ~3B active
toolsthinkingcode
The daily driver — only ~3B params active per token. On 24 GB the Q5 needed a light offload; on 32 GB it's fully resident, or drop to Q4 and spend the extra ~10 GB on a very deep context.
RTX 5090:Q5_K_M (26.5 GB) fully in VRAM — no offload on this card — or Q4_K_M (22 GB) with ~10 GB of KV headroom for very long contexts · 256K context
The dense sibling — slower than the MoE but rock-solid quality. At Q4 it uses barely half the card, which makes it the long-context workhorse: ~15 GB left over is a lot of KV cache.
RTX 5090:Q4_K_M (16.8 GB) fully in VRAM with ~15 GB free for KV cache — the deep-context pick · 256K context
The strongest local coding MoE. On 32 GB the low quant is nearly all-resident, and the better IQ4 needs only a light expert offload — a clear step up from what 24 GB cards manage.
RTX 5090:IQ3 (28 GB) nearly all-resident, or IQ4_XS (38 GB) with a light CPU-expert offload — auto-fit sets the split · large context
A frontier-class 120B reasoning MoE at native MXFP4. The ~63 GB of weights are mostly experts that park in system RAM — 32 GB of VRAM keeps more of them resident than any 24 GB card, so less traffic crosses the PCIe bus.
RTX 5090:MXFP4 (~63 GB), experts offloaded to CPU RAM with a lighter split than 24 GB needs — auto-fit sets it up · 128K context
The vision pick — Google's dense multimodal flagship with the stronger OCR the Qwen picks lack. On 24 GB the Q6 needed a light offload; here it's fully resident with room for images and context.
RTX 5090:Q6_K (25.2 GB) fully in VRAM — no offload, unlike 24 GB cards — with headroom for images + context · 256K context
The reach pick, borrowed from the hub's 48 GB tier — a 106B MoE with 12B active params for GLM's agentic/coding tier. On 32 GB it runs with its experts split to system RAM; expect it to lean on RAM bandwidth.
RTX 5090:Q2_K_XL (47 GB) or IQ3 (51 GB) with CPU-expert offload (needs system RAM) — auto-fit sets it up · 128K context
We won't print a made-up number for this card. The only speeds we've measured ourselves come from an
RTX 5070 Ti 16 GB (Qwen3.6-35B-A3B at a 200K context: 74.7 t/s) — a different GPU, so
don't extrapolate from it. What the community has published for the 5090 covers earlier-generation
analogues of the picks above:
Model benchmarked (earlier-gen analogue)
Quant
Community-reported generation speed
Qwen3 30B-A3B (MoE, ~3B active)
Q4_K_XL
~234 t/s at 4K context → ~110 t/s at 32K
Qwen3 30B-A3B, deep context
Q4_K_XL
~52 t/s with ~147K tokens of context fully in VRAM
Qwen3 32B (dense)
Q4_K_XL
~44–61 t/s (32K → 4K context)
Qwen3 14B (dense)
Q4_K_XL
~82–124 t/s
Qwen3 8B (dense)
Q4_K_XL
~112–186 t/s
Community-reported: hardware-corner.net's RTX 5090 llama-bench (llama.cpp) results at Q4_K_XL, 2025–2026. Qwen3.6-generation numbers for this card aren't broadly published yet.
Two patterns are worth pulling out. First, the MoE advantage: ~3B active params generate roughly 4×
faster than a dense 32B on the same card. Second, the context story — in that same testing the 30B MoE
held ~147K tokens of context entirely in VRAM while the dense 32B maxed out near 45K,
which is exactly why the extra 8 GB over a RTX 4090 matters more for long
sessions than for short ones. For gpt-oss-120B and GLM-4.5-Air with expert offload we
found no credible community numbers on the 5090 — speed there depends heavily on your system-RAM
bandwidth, so TurboLLM measures it on your machine at load time rather than guessing. The 16 GB
RTX 5080 drops to a different set of fits despite sharing the architecture.
How TurboLLM auto-fits the RTX 5090
Load any model and TurboLLM benchmarks it on your 5090, storing the measured tokens/sec
in the model list — live numbers from actual generation, not estimates. Before you load, every quant gets
a VRAM-fit verdict against your real 32 GB, including KV-cache growth at your chosen
context — on this card that's often the deciding number. Auto-fit then picks the GPU-layer count and the
MoE expert-offload split for you — it's what puts gpt-oss-120B's and GLM-4.5-Air's experts in system RAM
automatically. Details in the auto-tune docs.
FAQ
Can the RTX 5090 run a 70B model?
Closer than any other consumer card, but not fully in VRAM at a sensible quant — a dense 70B is
~43 GB at q4_K_M and ~38 GB at IQ4_XS, both over 32 GB. You can run it with
CPU offload (dense offload is slow) or at an aggressive ~2-bit quant. A 27B–35B model at q5–q6, or a
big MoE like gpt-oss-120B or GLM-4.5-Air with experts offloaded to system RAM, gives better results
on this card.
What quant should I use on 32 GB of VRAM?
q4_K_M is the default best quality-per-gigabyte, and on 32 GB it leaves enormous room
for context. The extra 8 GB over a 24 GB card is best spent stepping up: 27B–35B models run at
q5_K_M or q6-class fully in VRAM for near-lossless quality. TurboLLM shows a VRAM-fit
verdict for each quant before you download — the naming is decoded in
Quantization explained.
How fast will Qwen3.6 35B-A3B run on the RTX 5090?
There are no broad community benchmarks for this exact model on the RTX 5090 yet. Its previous
generation, Qwen3 30B-A3B at a q4-class quant, is community-reported around 234 tokens/sec
at short context and ~110 tokens/sec at a 32K context on this card. TurboLLM measures the real number
on your machine the moment you load the model.
How much context can the RTX 5090 hold in VRAM?
In a community llama-bench run, the 32 GB held about 147K tokens of context for Qwen3
30B-A3B at q4 entirely in VRAM, still generating at ~52 tokens/sec; a dense 32B at the same quant
maxed out near 45K. Quantizing the KV cache to q8_0 stretches those numbers substantially
further. TurboLLM's VRAM-fit verdict computes it for your exact model, quant, and context before you load.
Is the RTX 5090 better than the RTX 4090 for local LLMs?
Yes, on both axes that matter: 32 GB vs 24 GB lets every 24 GB pick step up a quant or hold a much
longer context, and 1,792 GB/s of memory bandwidth vs 1,008 GB/s means dense-model generation speed
scales up by a similar factor. If the 5090 is out of budget, the
RTX 4090 runs the same shortlist one quant step lower.