Best local LLMs for the Radeon RX 7900 XTX (24 GB) — updated July 2026
AMD's RDNA3 flagship gives you 24 GB — the same capacity as an RTX 3090 or 4090, usually
at a noticeably lower price. That 24 GB holds a 30B-class dense model fully in
VRAM at q4–q5, and with expert offload it runs a 120B-class MoE
like gpt-oss-120B — attention and KV cache on the GPU, experts in system RAM. What it
honestly won't do: hold a dense 70B at a sensible quant (that still needs ~40 GB), or run
the CUDA-only stacks — vLLM, SGLang, and most speed-hack forks are NVIDIA-first. But
GGUF + llama.cpp, which is what actually matters on a single consumer
card, is first-class on AMD — the fits below are identical to the
RTX 3090's.
Numbers you can trust
TurboLLM measures real tokens/sec on your card and shows a
VRAM-fit verdict before you load — no guessed speeds in the app, ever.
Every speed on this page is labeled with its source: the only numbers we measured
ourselves came from a different card entirely (an NVIDIA RTX 5070 Ti 16 GB), and
everything quoted for the 7900 XTX is community-reported.
The shortlist below is the models hub's 24 GB tier, plus the 16 GB tier's dense pick at the higher quant this card allows, with the fit worked out for this card. Everything a 16 GB card runs also runs here — at a higher quant, longer context, or lighter offload.
Qwen3.6 35B-A3B
by Alibaba · GGUF
MoE · ~3B active
toolsthinkingcode
The daily driver for this card — only ~3B params active per token, so it's dramatically faster than a dense model of similar quality, and on 24 GB it sits fully in VRAM at Q4.
RX 7900 XTX:Q4_K_M (22 GB) fully in VRAM with room for context, or Q5_K_M (26.5 GB) + light expert offload for max quality · 256K context
The dense sibling — slower than the MoE but rock-solid quality. A 16 GB card has to squeeze it to Q3/IQ4; the 7900 XTX runs it at Q4 with ~7 GB left over for a long context.
RX 7900 XTX:Q4_K_M (16.8 GB) fully in VRAM with generous KV-cache headroom · 256K context
The strongest local coding MoE. Its experts live in system RAM while attention and KV stay on the GPU — 24 GB means a higher quant and a lighter offload than a 16 GB card needs.
RX 7900 XTX:IQ3–IQ4_XS (28–38 GB) with CPU-expert offload — auto-fit sets the split · large context
A frontier-class 120B reasoning MoE at native MXFP4. The ~63 GB of weights are mostly experts, so they park in system RAM while attention and KV stay in VRAM — plan on plenty of RAM.
RX 7900 XTX:MXFP4 (~63 GB), experts offloaded to CPU RAM — auto-fit sets it up · 128K context
The vision pick — Google's dense multimodal flagship with the stronger OCR the Qwen picks lack. Fits fully in VRAM at Q4 with room for images and context.
RX 7900 XTX:Q4_K_M (18.3 GB) fully in VRAM, or Q6_K (25.2 GB) + light offload · 256K context
We won't print a made-up number for this card. The only speeds we've measured ourselves come from an
NVIDIA RTX 5070 Ti 16 GB (Qwen3.6-35B-A3B at a 200K context: 74.7 t/s) — a different
card from a different vendor, so don't extrapolate from it. What the community has published for the
7900 XTX itself:
Model benchmarked
Quant
Backend
Community-reported generation speed
Llama 2 7B (llama.cpp's standard bench)
q4_0
Vulkan (RADV)
~167–195 t/s
Llama 2 7B (llama.cpp's standard bench)
q4_0
ROCm (HIP)
~130–170 t/s
Qwen3.5 35B-A3B (previous gen of the MoE pick)
q4_0
Vulkan
~95–105 t/s
Qwen3.5 35B-A3B (previous gen of the MoE pick)
q4_0
ROCm
~75–78 t/s
Qwen3.6 35B-A3B with NextN multi-token prediction
IQ4_XS-class
ROCm 7.2.3
~130 t/s (single report)
Community-reported, aggregated from llama.cpp's GitHub benchmark discussions (ROCm scoreboard #15021, Vulkan scoreboard #10879, issue #20934), a Hugging Face llama-bench thread on Qwen3.5-35B-A3B-GGUF, and a Level1Techs forum log, 2025–2026. The ranges span driver and ROCm-version differences — they are real, not measurement noise.
Two patterns are worth more than any single row. First, the MoE effect: ~3B active params means a
35B-class model generates at roughly 100 t/s where a similar-quality dense model would crawl — which
is why Qwen3.6 35B-A3B is the daily-driver pick. Second, the backend story on this card is genuinely
unsettled: ROCm has traditionally led prompt processing (~3,300–3,900 t/s
community-reported on the standard 7B bench), while recent Vulkan builds match or beat ROCm
at token generation — the opposite of what most people expect. For
gpt-oss-120B with expert offload we found no credible community numbers on this card —
speed there depends heavily on your system-RAM bandwidth, so TurboLLM measures it on your machine at
load time rather than guessing. Weighing neighbors: the RTX 3090 runs the
exact same shortlist with the CUDA ecosystem behind it, and the
RX 9070 XT is AMD's newer RDNA4 card at 16 GB — current-gen, but a
tier down in fits.
How TurboLLM auto-fits the RX 7900 XTX
On first run TurboLLM detects the card and auto-provisions a ROCm llama.cpp build on
supported GPUs — RDNA3 is — with Vulkan as the universal fallback, and a backend
picker that lets you switch between them at any time (see Run any engine).
Given how closely the two backends trade blows on this card, being one dropdown away from the other
one matters. Load any model and TurboLLM benchmarks it on your 7900 XTX, storing the
measured tokens/sec in the model list — live numbers from actual generation, not
estimates. Before you load, every quant gets a VRAM-fit verdict against your real
24 GB, including KV-cache growth at your chosen context, and auto-fit picks the GPU-layer count and
the MoE expert-offload split for you — it's what puts gpt-oss-120B's experts in system RAM
automatically. Details in the auto-tune docs.
Linux vs Windows on this card
ROCm is Linux-first, and on Linux it's now a clean install for RDNA3. On Windows the picture is
narrower: as of mid-2026 AMD's own compatibility docs say the full ROCm stack is not yet
supported on Windows (PyTorch-on-Windows ships with ROCm 7.2.1 components, and HIP-based
llama.cpp builds exist). The dependable Windows path is Vulkan — which on this
card is genuinely fast, not a consolation prize.
FAQ
Can the RX 7900 XTX run a 70B model?
Not comfortably. A dense 70B needs roughly 40 GB at q4_K_M, so on 24 GB you're limited to
aggressive ~2-bit quants or heavy CPU offload, and both cost real quality or speed. A 30B-class model
at q4–q5, or a big MoE like gpt-oss-120B with its experts offloaded to system RAM, gives better results
on this card.
Should I use ROCm or Vulkan on the RX 7900 XTX?
Try both — they're one dropdown apart in TurboLLM's backend picker. In community llama.cpp benchmarks
the two trade blows on this card: ROCm has traditionally led prompt processing, while recent Vulkan
builds match or beat it at token generation. Which one wins on your machine depends on your driver and
ROCm version — which is exactly why TurboLLM measures instead of guessing.
Does the RX 7900 XTX work for local LLMs on Windows?
Yes — via Vulkan, which runs on the standard Radeon driver and is genuinely fast on this card, not a
consolation prize. ROCm itself is Linux-first: as of mid-2026, AMD's own compatibility docs say the
full ROCm stack is not yet supported on Windows. HIP-based llama.cpp builds for Windows exist, but
Vulkan is the path we'd recommend there.
Is the RX 7900 XTX better than the RTX 3090 for local LLMs?
They fit exactly the same models — both have 24 GB and near-identical memory bandwidth (960 vs
936 GB/s). The 3090 buys you the CUDA ecosystem: vLLM, SGLang, and most community forks are
NVIDIA-first, and benchmark coverage is broader. The 7900 XTX is typically cheaper, comes new with a
warranty, and for GGUF + llama.cpp — what you'll actually run on a single 24 GB card — it's
first-class. If you already own either, run it.
How fast will Qwen3.6 35B-A3B run on the RX 7900 XTX?
Community reports put the previous generation, Qwen3.5 35B-A3B at q4_0, around 95–105
tokens/sec on Vulkan and 75–78 on ROCm, and one forum log shows Qwen3.6 35B-A3B reaching about 130
tokens/sec on ROCm 7.2.3 with NextN multi-token prediction enabled. TurboLLM measures the real number
on your machine the moment you load the model.