The base M4 runs a dense 14B fully in memory at Q4 — that's the realistic ceiling. The 27B and 35B-MoE picks from our models hub's 16 GB tier (mainstream discrete GPUs — RTX 4070 Ti SUPER / 4080 / 5070 Ti) don't carry over: unified memory has no separate system RAM to offload MoE experts into, so what doesn't fit in the pool doesn't run. And it's one pool — macOS and your apps share the same 16 GB, so plan on about 10–12 GB genuinely available for model weights plus KV cache.
TurboLLM benchmarks every model on your exact machine and shows a memory-fit verdict before you load — real measured tokens/sec, never a spec-sheet guess. Every speed on this page is labeled with its source.
On a discrete 16 GB card, the OS lives in system RAM and the GPU gets all 16 GB — and an MoE model can spill its idle experts into system RAM. On the M4, weights, KV cache, macOS, and your browser all share one 16 GB pool, and macOS caps how much of it the GPU may take. The upside: no PCIe transfers, and the memory the model does get is genuinely fast to access. The shortlist below is sized for the ~10–12 GB you can realistically use.
The daily driver for the M4 — a fast dense all-rounder with a toggleable thinking mode, small enough to leave real headroom for context and your other apps.
Q4_K_M (5.0 GB) — fits with ~5 GB+ to spare for KV cache · 128K contextunsloth/Qwen3-8B-GGUFThe quality ceiling for this machine — the biggest dense model that runs comfortably inside the usable budget. Noticeably stronger than the 8B when you can spare the memory.
Q4_K_M (9.0 GB) fits the usable budget · IQ4_XS (8.1 GB) for more context headroom · skip Q5_K_M+ (10.5 GB) unless little else is runningunsloth/Qwen3-14B-GGUFThe best dense coder at this size — the newest coding models are big MoEs that want far more memory than 16 GB unified can offer.
Q4_K_M (4.7 GB) with room for a real coding context · 32K contextQwen/Qwen2.5-Coder-7B-Instruct-GGUFThe vision pick — current-gen vision-language model with strong OCR and spatial reasoning at a footprint the M4 handles easily.
Q4_K_M (5.0 GB) — headroom for images + context, though long-context memory is tighter here than on a discrete 16 GB cardQwen/Qwen3-VL-8B-Instruct-GGUFA tiny, top-tier multilingual embedding model for local RAG — costs almost nothing alongside any chat pick above. Served on /v1/embeddings.
Q8_0 (~0.6 GB) — pairs with any chat model · 32K contextQwen/Qwen3-Embedding-0.6B-GGUF
Qwen3.6 27B (Q3_K_M, 13.6 GB) and Qwen3.6 35B-A3B
(Q3_K_M, 16.6 GB) headline our 16 GB tier, but that tier
assumes a discrete card with separate system RAM for MoE expert offload. On the M4 both
blow past the ~10–12 GB usable budget, and there's no second memory pool to spill into.
They want 24–32 GB of unified memory — or a discrete 16 GB card like the
RTX 4070 Ti.
The base M4's memory bandwidth is 120 GB/s, and token generation on Apple Silicon is bandwidth-bound — so bigger files generate proportionally slower. The official llama.cpp Apple Silicon benchmark thread has measured numbers for the base M4 (10-core GPU):
| Test model | Quant | Prompt processing | Generation |
|---|---|---|---|
| Llama 7B | Q4_0 | ~221 t/s | ~24 t/s |
| Llama 7B | Q8_0 | ~224 t/s | ~13.5 t/s |
Community-reported: official llama.cpp Apple Silicon benchmark thread (GitHub, ggml-org discussion #4167), base M4 / 10-core GPU, 2024–2026. Llama 7B test model — not the models above.
So Qwen3 8B at Q4_K_M should land a little under that ~24 t/s, and Qwen3 14B
proportionally lower again — but we won't print a made-up number for a file nobody measured.
Load it and TurboLLM shows the real tokens/sec for your M4, measured from actual
generation and remembered per model.
On first run, TurboLLM detects Apple Silicon and downloads a prebuilt Metal build of llama.cpp — nothing to compile. It also supports MLX as an engine kind, so you can run Apple's own framework side by side and keep whichever measures faster (see engines). Every model gets a memory-fit verdict before you load, and auto-tune benchmarks on load and derives fast defaults for your exact machine instead of generic flags.
No. A 70B model is about 43 GB on disk at Q4_K_M, and even extreme low-bit
quants stay well above what 16 GB of unified memory can hold once macOS takes its share.
The practical ceiling on an M4 16 GB is a dense 14B at Q4_K_M. For a local
70B you want 64 GB+ of unified memory or a multi-GPU rig.
Q4_K_M for anything up to 14B. Qwen3 14B at Q4_K_M is 9 GB, which
fits the roughly 10–12 GB the M4 can actually give the model; drop to IQ4_XS
(8.1 GB) if you want more room for context. Models of 8B and under fit at Q4_K_M
with plenty of headroom. New to quant names? Read
Quantization explained.
The official llama.cpp Apple Silicon benchmark thread reports the base M4 generating about
24 tokens/sec on a 7B model at Q4_0 (community-reported). Qwen3 8B at
Q4_K_M is slightly larger, so expect a little under that. TurboLLM measures the
real number on your machine when you load the model.
No. TurboLLM detects Apple Silicon on first run and downloads a prebuilt Metal build of llama.cpp automatically. It also supports MLX as an engine kind if you want Apple's own inference framework.
Both work in TurboLLM. llama.cpp's GGUF ecosystem has far more models and quants; MLX is Apple's own framework and is often very fast for generation on Apple Silicon. Load the same model on both and keep whichever measures faster on your machine — the measured tokens/sec settles it, not a benchmark blog.
One command — no Xcode, no Python, no compiling. Requires Node 22+.
New here? Start with the getting-started guide or Quantization explained. On a discrete card instead? See the picks for the RTX 4060 (8 GB) and the RTX 4070 Ti (12/16 GB).