TurboLLM documentation
TurboLLM is a local-LLM platform that runs any local LLM engine, auto-tuned to your GPU — with a polished web UI and an OpenAI/Anthropic-compatible API. Bring your own llama.cpp fork. No compiling. No Electron. No Python. Point Claude Code at your own machine in one command, fully offline.
That one command starts a local daemon, opens a browser UI at
http://127.0.0.1:6996, and serves your models over an API any tool can talk
to. It needs Node.js 22 or newer — nothing else.
Start here
Install & first run →
Requirements, the one command, and what happens on first launch (it downloads the right engine for your GPU).
Load your first model →
Reuse the GGUFs you already have, or browse & download from Hugging Face in-app — with a VRAM-fit verdict.
Run any engine →
The headline feature: swap in any llama.cpp fork, vLLM, MLX, and more — with a hardware-fit verdict per engine.
Auto-tuning & speed →
How TurboLLM benchmarks your GPU, derives fast launch flags, and shows real measured tokens/sec.
Schedule tasks with Routines →
A chat or Code task that runs on its own, on a timer — human-confirmed, pausable, with a full run history.
Build on the API →
OpenAI- and Anthropic-compatible endpoints, API keys, and a gateway that loads models on demand.
Connect your tools →
One-command setup for Claude Code, opencode, Kilo, Continue, Cline, and more — pointed at your own GPU.
What makes it different
- Any engine, including forks. No other local-LLM app lets you run whatever inference engine you want. The fastest community innovations land in forks first — TurboLLM treats the engine as a swappable component.
- Auto-tuned to your hardware. It benchmarks your exact GPU on load, derives fast defaults, and shows a VRAM-fit verdict before you load — no flag guessing.
- Real measured tokens/sec. The speed you see is measured on your machine from actual generation — not a spec-sheet guess.
- Offline-first & private. No account, no backend, no internet. Your prompts, chats, files, and keys never leave your machine.
Head to the Models hub for hand-picked models that fit your GPU
(8 GB / 16 GB / 24 GB), and read Quantization explained
to understand what Q4_K_M vs Q8_0 actually means before you download.