A hand-picked shortlist by how much memory your GPU has — from an 8 GB laptop to a 128 GB DGX Spark — with the exact quant and offload that makes each one fit. Bigger models fit smaller boxes than you'd think: for an MoE, only a few billion params are active per token, so it runs with its experts in system RAM. New to quants? Read Quantization explained.
We don't print guessed speeds here. When you load one of these, TurboLLM benchmarks it on your exact GPU and shows the real measured tokens/sec plus a VRAM-fit verdict — the one thing Ollama's and LM Studio's model lists can't tell you.
The size next to each model is the on-disk GGUF at a given quant, and it's a dial, not a fixed
cost. Drop from Q4_K_M to Q3_K_M or an IQ3 and a 35B fits
16 GB; for a Mixture-of-Experts model, offload the idle experts to CPU RAM and only the active
~3B stay on the GPU. You don't have to work any of this out — TurboLLM's
auto-fit picks the quant and the exact GPU/CPU split for your card, and the
VRAM-fit verdict shows it before you load.
These run fully in VRAM at Q4. The newest flagships are bigger MoE models — with aggressive expert-offload even those can run here, but they shine on 16 GB+.
A fast, capable dense all-rounder with a toggleable thinking mode — the sweet spot for a small card.
Q4_K_M (5.0 GB) fully in VRAM · 128K contextunsloth/Qwen3-8B-GGUFThe vision pick for 8 GB — Google's newest small Gemma takes image input and still fits comfortably.
Q4_K_M (~5 GB) in VRAM · 128K contextunsloth/gemma-4-E4B-it-GGUFStill the best dense coder that fits 8 GB fully — the newest coders are big MoE models that want 16 GB+.
Q4_K_M (4.7 GB) in VRAM · 32K contextQwen/Qwen2.5-Coder-7B-Instruct-GGUFA tiny, top-tier multilingual embedding model for local RAG — costs almost no VRAM alongside a chat model. Served on /v1/embeddings.
Q8_0 (~0.6 GB) — pairs with any chat model · 32K contextQwen/Qwen3-Embedding-0.6B-GGUFThe sweet spot. A 27B dense fits in VRAM at Q3–IQ4; a 35B-A3B MoE runs at Q3 in-VRAM or Q4 with expert offload; even a huge coder MoE runs with its experts in system RAM.
The fast MoE daily driver — only ~3B params active per token, so it stays quick even with experts spilled to CPU. This is the model behind our RTX 5070 Ti 16 GB benchmarks.
Q3_K_M (16.6 GB) in VRAM, or Q4_K_M (22 GB) + expert offload · 256K contextunsloth/Qwen3.6-35B-A3B-GGUFThe dense sibling — a touch slower than the MoE but rock-solid quality, and it fits fully in VRAM at a mid quant.
IQ4_XS (15.4 GB) or Q3_K_M (13.6 GB) in VRAM · Q4_K_M (16.8 GB) + light offload · 256K contextunsloth/Qwen3.6-27B-GGUFThe newest agentic coding MoE. Its experts live in system RAM while attention and KV stay on the GPU, so it runs on a 16 GB card — TurboLLM's auto-fit sets the split.
IQ2–IQ3 with heavy CPU-expert offload (needs system RAM) · even better on 24 GB+ · large contextunsloth/Qwen3-Coder-Next-GGUFCurrent-gen vision-language model with strong OCR and spatial reasoning; a tiny footprint leaves lots of VRAM for images and context.
Q4_K_M (5.0 GB) with plenty of headroom for images + context · 256K contextQwen/Qwen3-VL-8B-Instruct-GGUFEverything in the 16 GB tier runs here at a higher quant, longer context, or lighter offload — plus the bigger flagships below.
The same daily driver, now fully in VRAM at Q4 — faster and higher quality than on a 16 GB card.
Q4_K_M (22 GB) fully in VRAM with room for context, or Q5_K_M (26.5 GB) + light offload for max quality · 256K contextunsloth/Qwen3.6-35B-A3B-GGUFMore VRAM means a higher quant and a smaller offload footprint — the strongest local coder gets noticeably better here.
IQ3–IQ4_XS (28–38 GB) with lighter offload than 16 GB · large contextunsloth/Qwen3-Coder-Next-GGUFA frontier-class 120B reasoning MoE at native MXFP4. Keep attention and KV in VRAM, park the experts in system RAM, and it runs on a single 24 GB card.
MXFP4 (~63 GB), experts offloaded to CPU RAM — auto-fit sets it up · 128K contextggml-org/gpt-oss-120b-GGUFGoogle's dense multimodal flagship — image + text input and the stronger vision/OCR the Qwen picks lack.
Q4_K_M (18.3 GB) fully in VRAM, or Q6_K (25.2 GB) + light offload · 256K contextunsloth/gemma-4-31B-it-GGUFDense 70B models fit fully in VRAM, and the 100B-class MoEs start to open up at a low quant.
The classic dense 70B — excellent quality and the widest ecosystem support. Being dense, it likes memory bandwidth, so it's happiest on a discrete card.
Q4_K_M (43 GB) fully in VRAM · IQ4_XS (38 GB) · 128K contextunsloth/Llama-3.3-70B-Instruct-GGUFThe agentic coder at a proper quant — this is where it really comes into its own, with most of it resident instead of offloaded.
Q4_K_M (48.5 GB) with a touch of offload, or Q3 fully in VRAM · large contextunsloth/Qwen3-Coder-Next-GGUFA 106B-A12B MoE — the smallest way into GLM's agentic/coding tier. Fits 48 GB at a low quant; noticeably better on 128 GB.
Q2_K_XL (47 GB) fits, or IQ3 (51 GB) + light offload · 128K contextunsloth/GLM-4.5-Air-GGUFYour 16 GB daily driver, now at Q8 — near-lossless quality, fully resident, with headroom for a huge context.
Q8_0 (37 GB) fully in VRAM · 256K contextunsloth/Qwen3.6-35B-A3B-GGUFDGX Spark's 128 GB is unified memory at modest bandwidth, so low-active-param MoE models are the sweet spot — they run huge and stay fast. Everything from the lower tiers also runs here at full quality with no offload (gpt-oss-120b, Qwen3-Coder-Next at Q8, …).
The open-weight flagship — 235B total, 22B active. On a DGX Spark it runs at a usable quant with room to spare; genuine frontier quality, fully local.
Q3_K_M (112 GB) or IQ4_XS (125 GB) · 256K contextunsloth/Qwen3-235B-A22B-Instruct-2507-GGUFThe vision flagship at the same scale — frontier multimodal reasoning and OCR, running entirely on your own hardware.
Q3_K_M (112 GB) / IQ4_XS (125 GB) · vision + 256K contextunsloth/Qwen3-VL-235B-A22B-Instruct-GGUFA fast agentic/coding MoE — only ~10B params active, so it stays brisk even on unified memory. A strong coding and tool-use choice at this scale.
Q3_K_M (101 GB) / IQ4_XS (108 GB) · large contextunsloth/MiniMax-M2.7-GGUFThe 106B GLM at Q8 — DGX Spark holds it entirely in memory for near-lossless agentic and coding quality.
Q8_0 (117 GB) fully resident · or Q6_K (99 GB) · 128K contextunsloth/GLM-4.5-Air-GGUFThe very largest open MoEs — Qwen3-Coder-480B, DeepSeek-V3, Kimi-K2 — need roughly 192 GB+ even at a low quant. Two DGX Sparks linked over ConnectX give you 256 GB of unified memory, enough to run a 400B-class model locally.
Open-weight models move fast, so this page is a hand-picked starting point (refreshed mid-2026). Inside TurboLLM, Models → Browse Hugging Face is a live, sortable catalog of the entire Hub with a rendered model card, per-GPU quant recommendation, and resume + SHA-256 verified downloads. To add any model above: copy its repo id, then paste it into that in-app search.
Q4_K_M? IQ3_XXS? Q8_0? The naming is the most confusing part of local LLMs, and nobody explains it. We do.