How to Use TurboLLM with Kaggle

What you actually get

A free Kaggle account includes roughly 30 GPU-hours per week and, under Accelerator, a GPU T4 × 2 option: two Tesla T4 cards, 16 GB each, about 30 GB pooled. That is more VRAM than most consumer desktops have, and it is the cheapest way to find out what a dual-GPU machine feels like before buying into one.

What runs on it is not a stripped-down demo. It's the same TurboLLM daemon you'd run locally: the model manager, auto-tune, Chat, Code, and the OpenAI- and Anthropic-compatible APIs — served over a public tunnel, so you open it in a normal browser tab instead of squinting at notebook output.

Step 1 — Open and copy the notebook

Start from the published notebook rather than an empty one; it carries the preflight checks and the setup script:

kaggle.com/code/sonijisons/turbollm-one-click-dual-t4

Click Copy & Edit to get your own editable copy. You need a Kaggle account, and it must be phone-verified — Kaggle gates both GPU access and notebook internet access behind verification, and without it the run stops at the very first cell.

Step 2 — Set the three session settings

In the notebook's right-hand Settings panel:

• Accelerator → GPU T4 × 2. Not "GPU T4 ×1", not P100. The dual shape is the whole point, and Kaggle can't pin the number of cards from notebook metadata — it has to be chosen here.
• Internet → On. The notebook clones the repo, installs Node dependencies, and opens the tunnel. All three need outbound network.
• Add Input → Datasets → turboquant-cuda-t4. The prebuilt CUDA engine. Already attached on the published notebook — just confirm it survived the copy.

The first cell checks all three and fails immediately if any is wrong, so a misconfigured session costs you seconds rather than five minutes.

Step 3 — Run All, then open the printed link

The run installs Node and the daemon dependencies, builds the web UI, unpacks the CUDA engine into /tmp, starts the daemon, and opens a public *.trycloudflare.com tunnel. Roughly five minutes the first time. The cells are idempotent, so re-running is cheap.

The final cell prints exactly what you need:

============================================================
  OPEN YOUR TURBOLLM GUI
============================================================
  URL  : https://<random-words>.trycloudflare.com
  Token: <access token>
============================================================

Open the URL, paste the token, and you're in the normal TurboLLM UI. Copy the token from the notebook's text output rather than a screenshot — l/1 and B/8 are easy to misread, and a wrong character just looks like a rejected login.

Step 4 — Get a model in

The published notebook already ships one attached — Qwen3.8-27B UD-Q4_K_XL, mounted read-only from /kaggle/input and registered automatically, so Run All lands you on a working model with nothing to download. To use a different one, there are two routes, and the first is much better on Kaggle:

• Attach a public Kaggle dataset that already holds the GGUF. Datasets mount read-only at /kaggle/input essentially instantly and — the part that matters — don't count against the 19.5 GB /kaggle/working quota. A 22 GB model costs you nothing. The notebook registers those directories automatically.
• Download inside the UI via Models → Download. Convenient, but it lands in /kaggle/working, so you spend both quota and wall-clock, and it's gone when the session ends.

Public GGUF datasets that suit the dual-T4 shape:

tahsinekajolasalami/qwen36-35b-a3b-gguf   Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf   22.9 GB
selahattinkabasakal/qwen3-8-gguf          Qwen3.8-27B-UD-Q4_K_XL.gguf       17.9 GB

Step 5 — Auto-tune, then chat

Run auto-tune on the model before chatting. On a dual-GPU box it's doing more than picking a layer count: it projects VRAM per card and derives a byte-balanced split across the two T4s, so a model too big for one card spreads across the pooled 30 GB instead of piling onto one while the other sits nearly empty.

That distinction is worth spelling out, because it's the failure mode dual-GPU machines hit in general. A MoE model with CPU expert offload strips the experts out of its first N layers, leaving those layers roughly 10× lighter than the rest — while llama.cpp's default split divides purely by layer count. The result is wildly uneven: measured on this exact hardware, 1.7 GB on one card against 14.7 GB on the other, using barely half of the 30 GB pool. The derived split fixed that run to 14.9 / 12.8 GB — 90% of the pool — and chat went from 4.3 to 10.0 tok/s.

Measured end to end on the free 2×T4 shape: Qwen3.6-35B-A3B UD-Q4_K_XL (22.9 GB) at a 32k context, both cards loaded and no CPU offload at all, streams chat at ~25 tok/s. Prefill runs around 839 tok/s, so with 24k tokens already in the window it's time-to-first-token, not decode speed, that you actually wait on.

Better: use it from your own machine, with Turbo Link

Opening a tunnel URL in a browser tab works, but it isn't where you actually live. With Turbo Link you can point your local TurboLLM at the Kaggle box and have its models appear in your own model picker, prefixed with the machine's name — same UI, same chats, same API, except the generation happens on two free T4s instead of your GPU.

1. On the Kaggle box, turn on Settings → Experimental → Turbo Link, name the machine (e.g. kaggle-t4), and mint an Inference only link. Copy the tllink_… string — it's shown once.
2. On your own machine, turn the same flag on and paste the string into Settings → Turbo Link.
3. The Kaggle models now show up locally as kaggle-t4/<model> — in the chat picker, in /v1/models, and in turbollm launch --model.

Because a Kaggle session hands you a new tunnel URL each time, edit the link's Base URL on the next run instead of minting a fresh link. Turbo Link is experimental and off by default on both ends — the Turbo Link docs cover the grant presets and the security model.

Living with Kaggle's limits

• Sessions are temporary. Nothing outside /kaggle/working survives, and even that is capped at 19.5 GB. /tmp has about 1 TB but vanishes entirely — which is exactly why the notebook unpacks the 3.8 GB engine there.
• GPU time is quota'd at roughly 30 hours a week. Stop the session when you're done; an idle notebook still burns it.
• The tunnel is tied to the session. If it drops, re-run the launch cell for a fresh URL and token.
• Kaggle mangles the engine dataset — it strips executable bits and stores symlinks as 0-byte files. The setup script repairs both. If you drive llama-server by hand instead, you'll have to replay that repair yourself.

Running it headless from the CLI

The UI flow needs a human clicking through settings. For an unattended, reproducible run, push the notebook as a batch kernel instead:

pip install kaggle
kaggle auth login
kaggle kernels push -p deploy/kaggle --accelerator NvidiaTeslaT4
kaggle kernels status <owner>/<slug>

Three things that cost real time to learn:

• --accelerator NvidiaTeslaT4 is what pins the dual-T4 shape. enable_gpu in kernel-metadata.json is a bare boolean and cannot request two cards; the CLI flag can.
• The kernel slug comes from title, not id. A mismatch pushes once and then fails with a confusing 409 or a permission error. Keep the two identical.
• A batch kernel needs a keep-alive or the session ends the instant the last cell returns, taking the tunnel with it. The notebook's final cell blocks for up to nine hours, but only when it detects a batch run — Run All in the editor still finishes normally.

When Kaggle is, and isn't, the right answer

It's excellent for trying a model class your own hardware can't hold, for seeing how dual-GPU placement behaves before buying a second card, and for showing someone TurboLLM without asking them to install anything. It is not a place to keep a model — sessions end, storage resets, and a public tunnel isn't a deployment. For anything ongoing, run TurboLLM locally:

One command, no GPU rental, and the same UI at http://127.0.0.1:6996. See Install & first run to start there instead.