Run any engine
No other local-LLM app lets you run whatever inference engine you want. TurboLLM treats the engine as a swappable component — this is the headline feature. The fastest community innovations land in forks first, and TurboLLM lets you use them the day they ship.
Engine types
These are all first-class engine kinds — install from the curated catalog, pick the right one per model, and switch from a single dropdown:
- llama.cpp / GGUF — the official ggml-org/llama.cpp, auto-provisioned for your GPU on first run, plus curated forks ik_llama.cpp and TurboQuant.
- KoboldCpp and llamafile — GGUF, every OS.
- MLX — Apple Silicon.
- vLLM and SGLang — Linux + NVIDIA, for safetensors and high-throughput serving.
The Engines screen shows a hardware-fit verdict and grounded pros/cons for each before you install anything. Every engine runs under a real state machine: health-gated readiness, graceful stop, an idle auto-stop watchdog, and live logs + clear error surfacing in the UI when something fails to load.
Don't want to build anything?
On first run TurboLLM downloads the right upstream prebuilt for your GPU automatically. A backend picker then lets you switch between CUDA / ROCm / Metal / SYCL / Vulkan / CPU at any time — it downloads the variant you choose, LM Studio-style.
CUDA for NVIDIA, ROCm for AMD, Metal for Apple, SYCL for Intel, Vulkan otherwise — with a CPU fallback so it always runs, even without a GPU.
Add your own engine
Engines screen → Add your own engine. Compile or download any
llama-server-compatible binary — stock llama.cpp, a community fork, or your own
build — then:
-
Point TurboLLM at the folder
It scans for the
llama-serverbinary, runs a capability probe, and learns exactly which flags and features that build supports. Optionally paste the source repo URL so TurboLLM flags when a newer build ships. -
Activate it
The load-parameter UI adapts to that engine — features the build doesn't support are hidden; ones it adds (e.g. low-bit KV cache, NextN) light up.
Build from a git repo, in-app
Engines → Add via git repo lets you point at any llama.cpp-compatible fork's URL (and optional branch) and build it in-app with one click, reusing the existing build pipeline — no manual cloning and pointing at a binary by hand.
The build-from-source guide checks your toolchain (git / CMake / CUDA / a compiler — MSVC on Windows, gcc/clang on Linux), hands you the exact build commands (or a one-click "Build it for me" on Windows and Linux), then drops you into the folder scan above.
Sharing the GPU with ComfyUI
Running image generation on the same GPU? TurboLLM yields automatically:
- The instant ComfyUI starts a render, TurboLLM unloads its model and pauses new loads.
- When ComfyUI's queue drains, TurboLLM reloads the exact model it unloaded.
- Push-based, not polling — ComfyUI signals TurboLLM the moment a job starts or ends.