Best local LLMs for the Radeon RX 9070 XT (16 GB) — updated July 2026
The RX 9070 XT is AMD's current-generation 16 GB card, and 16 GB is the local-LLM sweet
spot. A dense 27B fits fully in VRAM at IQ4_XS, and a
35B-A3B MoE runs at Q3 with a light expert nudge to system RAM, or Q4 with fuller
expert offload — an offload trick that scales up to a much bigger MoE coder if you
have the RAM for it.
What it honestly won't do: a dense 70B, which needs ~40 GB at a sensible quant. The
fits here are the same as any 16 GB card — the
RTX 5060 Ti runs the identical shortlist — what's different
on RDNA 4 is the backend story: Vulkan works out of the box, ROCm support
is now official but arrived well after the card did, and community numbers currently favor
Vulkan for generation on this GPU.
Numbers you can trust
TurboLLM measures real tokens/sec on your card and shows a
VRAM-fit verdict before you load — no guessed speeds in the app, ever.
Every speed on this page is labeled with its source: the only numbers we measured ourselves
came from a different card entirely (an NVIDIA RTX 5070 Ti 16 GB), and everything quoted
for the 9070 XT is community-reported.
The shortlist below is the models hub's 16 GB tier with the fit worked out for this card, plus the 8 GB tier's dense all-rounder at the near-lossless quant this card allows.
Qwen3.6 35B-A3B
by Alibaba · GGUF
MoE · ~3B active
toolsthinkingcode
The MoE daily driver for 16 GB — only ~3B params active per token, so it stays quick even when some experts spill to system RAM. The best quality-per-watt-of-patience on this card.
RX 9070 XT:Q3_K_M (16.6 GB) just spills past 16 GB, so auto-fit nudges a few experts to system RAM — or Q4_K_M (22 GB) + expert offload · 256K context
The dense sibling — a touch slower than the MoE but rock-solid quality, and it fits fully in this card's VRAM at a mid quant. Dense models love memory bandwidth, and ~645 GB/s is plenty.
The newest agentic coding MoE. Its experts live in system RAM while attention and KV cache stay on the GPU, so it runs on a 16 GB card — TurboLLM's auto-fit sets the split.
RX 9070 XT:IQ2–IQ3 with heavy CPU-expert offload (needs system RAM) · even better on 24 GB+ · large context
The vision pick — current-gen vision-language model with strong OCR and spatial reasoning. Its tiny footprint leaves most of the 16 GB free for images and context.
RX 9070 XT:Q4_K_M (5.0 GB) with plenty of headroom for images + context · 256K context
The 8 GB tier's dense all-rounder, promoted — on 16 GB it runs at near-lossless quants with a long context, and it's the fastest thing on this list when you just want instant answers.
RX 9070 XT:Q6_K (6.7 GB) or Q8_0 (8.7 GB) fully in VRAM — near-lossless · 128K context
We won't print a made-up number for this card. The only speeds we've measured ourselves come
from an NVIDIA RTX 5070 Ti 16 GB (Qwen3.6-35B-A3B at a 200K context:
74.7 t/s) — a different GPU on a different backend, so don't extrapolate from it. What the
community has published for the 9070 XT is mostly backend shoot-outs, and the backend
is the story on RDNA 4:
What was measured
Backend
Community-reported speed
7B Q4_0 — llama-bench generation (tg128)
Vulkan
~137–138 t/s
7B Q4_0 — llama-bench generation (tg128)
ROCm (HIP)
~97–101 t/s
7B Q4_0 — llama-bench prompt processing (pp512)
Vulkan and ROCm
~4,900–5,050 t/s (a wash)
Qwen3.5 9B Q6_K_XL — generation
Vulkan (llama.cpp)
~62 t/s
Community-reported: rows 1–3 from knightli.com's llama.cpp CUDA/ROCm/Vulkan scoreboard (aggregating the llama.cpp GitHub benchmark discussions, April 2026); row 4 from Ivan Angelov's RDNA4 benchmark write-up at digtvbg.com (2026), where the same model on vLLM/ROCm managed 48 t/s. Numbers for the Qwen3.6-generation picks above aren't broadly published for this card yet.
Two honest take-aways. First, Vulkan currently generates ~35% faster than ROCm on
this card in the standard llama.cpp benchmark, while prompt processing is
essentially tied — a software gap, not a hardware one, and it shifts with every release, so
measure both. Second, against its AMD neighbor: in the same scoreboard the
RX 7900 XTX generates faster (~195 vs ~138 t/s on Vulkan —
it has 960 GB/s of bandwidth to this card's ~645) while the 9070 XT processes prompts
noticeably faster (~5,000 vs ~3,500 t/s). Older card, more bandwidth, more VRAM; newer
card, faster prefill. TurboLLM measures your actual numbers either way.
How TurboLLM auto-fits the RX 9070 XT
Load any model and TurboLLM benchmarks it on your 9070 XT, storing the
measured tokens/sec in the model list — live numbers from actual
generation, not estimates. Before you load, every quant gets a
VRAM-fit verdict against your real 16 GB, including KV-cache growth at
your chosen context. Auto-fit then picks the GPU-layer count and the MoE expert-offload
split for you — it's what makes Qwen3-Coder-Next usable on this card. And because the
backend matters so much on RDNA 4, the backend picker downloads the
Vulkan or ROCm engine variant with one click so you can compare measured speeds instead of
trusting a benchmark from someone else's box — see Run any engine
and the auto-tune docs.
FAQ
Does ROCm officially support the RX 9070 XT?
Yes — now. RDNA 4 had no official ROCm support on launch day; it arrived months later in
ROCm 6.4.1, and current ROCm 7.x releases list the RX 9070 XT (gfx1201) as officially
supported on Linux and Windows. Vulkan never needed any of that — it works with the
standard graphics driver, and in community llama.cpp benchmarks it is currently the faster
backend for generation on this card.
Is Vulkan or ROCm faster for local LLMs on the RX 9070 XT?
Community llama-bench numbers put Vulkan ahead for generation on this card — roughly 137 vs
101 tokens/sec on the standard 7B Q4_0 test — while prompt processing is a wash
at around 5,000 tokens/sec on both. That gap is software, not hardware, and it moves with
every driver and llama.cpp release, so TurboLLM lets you install both backends and measures
each one on your machine.
What is the biggest model an RX 9070 XT can run?
A 27B dense model fits fully in the 16 GB at IQ4_XS, and Qwen3.6 35B-A3B — a
MoE with only ~3B active params — runs at Q3 with a few experts nudged to system RAM, or
Q4 with fuller expert offload. With heavier expert offload a big MoE coder like
Qwen3-Coder-Next is genuinely usable, as long as you have the system RAM for the idle experts.
Can the RX 9070 XT run a 70B model?
Not sensibly. A dense 70B needs roughly 40 GB at q4_K_M, so 16 GB would force
extreme quantization plus heavy offload, and quality and speed both fall off a cliff. A 27B
dense at IQ4_XS or the 35B-A3B MoE gives far better results on this card —
the quant trade-offs are decoded in Quantization explained.
Do I need to install ROCm myself to run local LLMs on the RX 9070 XT?
No. TurboLLM detects the card and downloads a prebuilt engine automatically — nothing to
compile. A backend picker then lets you switch between ROCm and Vulkan at any time; Vulkan
works with the regular AMD graphics driver, and community benchmarks currently put it ahead
for generation on this card, so it is worth measuring both.
Try it on your 9070 XT
$npx turbollm
One command: it detects the card, downloads an engine, and opens the UI. New here? Start
with Install & first run and
Quantization explained. On different hardware, see the
RX 7900 XTX (AMD's 24 GB option) or the
RTX 5060 Ti (NVIDIA's budget 16 GB) guides.