Home / Models / Qwen3-Coder-Next

How to Run Qwen3-Coder-Next Locally — Setup Guide (August 2026)

Qwen3-Coder-Next is Alibaba's newest agentic coding model — an 80B-parameter Mixture-of-Experts with only ~3B active per token, built for long tool-use and coding sessions rather than one-shot answers.

Numbers you can trust

We don't print guessed speeds. Every measured number on this page came from TurboLLM's own auto-tuner running Qwen3-Coder-Next on real hardware — labeled as such. When you run it yourself, TurboLLM benchmarks it on your exact GPU and shows a VRAM-fit verdict before you load, and the real measured tokens/sec once it's running.

Does it fit your GPU?

It uses a linear-attention design (48 layers) instead of standard transformer attention, which is why its generation speed stays flat as a conversation gets longer instead of slowing down with depth the way a standard transformer does.

Qwen3-Coder-Next
Alibaba · GGUF
MoE · agentic coder · ~3B active
codetools
VRAMFit
16 GBIQ2–IQ3 (~43.7 GB on disk at MXFP4) with heavy CPU-expert offload — needs roughly 44 GB of free system RAM for the offloaded experts · large context
24 GB+IQ3–IQ4_XS (28–38 GB) with a lighter offload split · large context
48 GBQ4_K_M (48.5 GB) with a touch of offload, or Q3 fully in VRAM · large context
unsloth/Qwen3-Coder-Next-GGUF
View on Hugging Face →

How to run it

  1. Install TurboLLM

    npx turbollm — no install step, works on Windows, macOS, and Linux. It detects your GPU and auto-provisions a matching llama-server build (no CUDA toolkit, no Python, no compiler).

  2. Get the model

    Paste unsloth/Qwen3-Coder-Next-GGUF into TurboLLM's in-app Hugging Face search — it lists every quant with a VRAM-fit verdict against your real free VRAM, and downloads are resumable and SHA-256 verified.

  3. Let auto-fit pick the setup

    Flip the auto-fit toggle and TurboLLM chooses the GPU/CPU layer split (and, for a Mixture-of-Experts model, the expert-offload split) for your card — the same manual flag-hunting every other guide walks you through by hand.

  4. Load it

    TurboLLM benchmarks the model on your exact GPU as it loads and shows the real measured tokens/sec — not a number copied from someone else's card.

  5. Use it

    Chat in the built-in UI, or point any OpenAI- or Anthropic-compatible tool (including Claude Code) at TurboLLM's local API — see the API docs.

Measured, not guessed

RTX 5070 Ti 16 GBTurboLLM (measured)
Auto-tune result — nCpuMoe=39 (9 of 48 layers' experts on GPU), turbo2 KV, 200K ctx39.8 t/s generation
Decode at 393 tokens deep38.75 t/s
Decode at 1,417 tokens deep43.52 t/s
Decode at 5,577 tokens deep42.51 t/s
Decode at 22,153 tokens deep43.24 t/s

Measured via TurboLLM's auto-tune on an RTX 5070 Ti 16 GB — roughly 15.4 GB VRAM used, 375 MB of headroom in a 16 GB budget. July 2026.

One honest caveat

The one honest caveat is time-to-first-token, not generation speed: prefill runs through the CPU-resident experts, so a short prompt takes roughly 7 seconds, a 5.5K-token prompt about 16 seconds, and a 22K-token prompt about 56 seconds before the first token appears. Generation itself then holds a flat ~40 t/s regardless of depth. Budget system RAM too — the offloaded experts need roughly 44 GB free alongside the model.

FAQ

Why does Qwen3-Coder-Next suddenly get slow or stutter?

The usual cause is silent VRAM spill: once a model's weights plus its KV cache stop fitting in VRAM, the driver quietly offloads the overflow to system RAM over PCIe, and that path runs 5–10× slower with no error message — just a sudden slowdown. TurboLLM's auto-fit sizes the quant and GPU/CPU split against your card's real free VRAM (including KV-cache growth at your chosen context) specifically to avoid this, and shows a VRAM-fit verdict before you load rather than after it's already crawling. Full mechanics in VRAM spill explained.

Which quant should I use for Qwen3-Coder-Next?

Pick the highest quant that fits your VRAM with room left over for the KV cache and compute buffer — not just the model weights. As a starting point, Q4_K_M is usually the sweet spot (small quality loss for a large size drop); step up to Q5_K_M/Q6_K/Q8_0 if you have headroom, or down to Q3_K_M/an IQ quant if you don't. TurboLLM shows a VRAM-fit verdict for every quant against your card's real free VRAM before you download — see Quantization explained for what the letters and numbers actually mean, or llama.cpp flags that actually matter for the other settings worth knowing.

Will Qwen3-Coder-Next run on my GPU?

Check the VRAM table above for the tier closest to your card. If you're not sure which tier you're in, or want picks tailored to your exact hardware, use What can I run? — enter your VRAM (or Apple unified memory) and it suggests the model, quant, and offload split that fits, the same logic TurboLLM's auto-fit runs when you load a model.

Run it now

$ npx turbollm

One command detects your GPU, provisions the right engine, and opens the UI. New here? Start with Install & first run and Quantization explained. On different hardware, see RTX 5070 Ti, RTX 4090 or RTX 3090. Also see how to run it.