External Chat API

A public, versioned, tenant-aware REST + SSE API for building real chat apps on top of TurboLLM — with persistence, history, editing, and (adapter-permitting) branching built in. An integrating app implements exactly two things — send a message and consume an SSE stream — and never writes a conversation schema of its own.

What this is

TurboLLM already has a private, undocumented chat store behind its own web UI: conversations, messages, editing, branching, folders. This API promotes that capability into a stable, versioned, public surface at /api/ext/v1, so any application can be a chat app without first building and operating its own conversation database — schema, migrations, message ordering, attachments, token accounting.

This is not the gateway

TurboLLM also serves OpenAI- and Anthropic-compatible endpoints at /v1. That gateway is stateless — you send the full message history on every request and TurboLLM remembers nothing. Pick whichever fits:

Enable it

The external API is off by default — it exposes tenant data, so turning it on is an explicit, deliberate step, never something a version bump switches on.

1. Turn it on

   Set api.ext.enabled: true in config.json. While off, every route under /api/ext/v1 answers a plain 404 — there's no reason to confirm the surface exists to someone who can't use it.

   {
     "api": {
       "ext": {
         "enabled": true,
         "maxInFlightPerTenant": 4,
         "requestsPerMinutePerTenant": 120
       }
     }
   }

   Copy

2. Mint a tenant key

   Every request is scoped to a tenant — the integrating application — resolved from the presented key and never from anything in the request itself. A key belongs to exactly one tenant and carries coarse scopes (chats:read, chats:write, runs:write). There's no dedicated key-minting UI for this yet, so add one directly to config.json's apiKeys array — the daemon only ever stores a key's SHA-256 hash, never the raw value:

   // Compute the hash for a secret you generate yourself, e.g.:
   node -e "console.log(require('crypto').createHash('sha256').update('YOUR-SECRET-HERE').digest('hex'))"

   Copy

   {
     "apiKeys": [{
       "id": "...", "name": "my-integration", "prefix": "tllm-ext-...",
       "hash": "<sha256 hex from above>",
       "tenant": "my-integration",
       "scopes": ["chats:read", "chats:write", "runs:write"],
       "createdAt": "2026-08-18T00:00:00.000Z", "lastUsedAt": null
     }]
   }

   Copy

3. Point your app at the base URL

   Same host and port as everything else TurboLLM serves:

   $http://localhost:6996/api/ext/v1Copy

Five-minute quickstart

Every mutating call is scoped by owner — your integration's own end-user id, opaque to TurboLLM (defaults to "default" if you have only one). Here's create a chat, send a message, and consume the stream — TypeScript client and raw curl side by side.

import { TurboLLMChat } from "@turbollm/chat-client";

const client = new TurboLLMChat({
  baseUrl: "http://localhost:6996/api/ext/v1",
  apiKey: process.env.TURBOLLM_TENANT_KEY!,
});

const chat = await client.chats.create({ title: "Support thread", owner: "user_4821" });

const stream = client.send(chat.id, {
  owner: "user_4821",
  content: "Summarize the attached report.",
});

for await (const ev of stream) {
  if (ev.event === "delta") process.stdout.write(ev.data.content);
}

// stream.outcome is 'complete' | 'failed' | 'aborted' | 'unknown' once the loop ends —
// see Resumption below for what 'unknown' means and why it's not an error.
console.log("\n\noutcome:", stream.outcome, "last event_seq:", stream.lastEventSeq);

# 1. Create a chat
curl -s http://localhost:6996/api/ext/v1/chats \
  -H "Authorization: Bearer $TURBOLLM_TENANT_KEY" \
  -H "Content-Type: application/json" \
  -d '{"title":"Support thread","owner":"user_4821"}'
# -> { "id": "chat_01J8...", "owner": "user_4821", "title": "Support thread", ... }

# 2. Send a message and stream the reply
curl -N http://localhost:6996/api/ext/v1/chats/chat_01J8.../messages \
  -H "Authorization: Bearer $TURBOLLM_TENANT_KEY" \
  -H "Idempotency-Key: $(uuidgen)" \
  -H "Accept: text/event-stream" \
  -H "Content-Type: application/json" \
  -d '{"role":"user","content":"Summarize the attached report.","owner":"user_4821","generate":true}'
# -> event: run   / event: delta (repeated) / event: usage / event: done

The TypeScript client's send()/resume() surface implements the reconciliation rule in the next section automatically. If you write your own SSE client, you need to implement it yourself — it's the one rule that actually matters here.

Resumption: the one rule that matters

The resume token is event_seq — a single counter across every event kind (delta, reasoning, tool_call, usage), sent as the SSE frame's id: field. Reattach to a run with ?after=<event_seq>:

GET /api/ext/v1/runs/run_01J8.../stream?after=411

• after within the run's replay buffer (4 MB or 5,000 events, whichever is smaller) → exact replay from that point, then live, no gap and no duplicate.
• after older than the buffer's oldest event → 409 replay_window_exceeded. Re-read the message via GET /messages/{id} to resynchronize, then reattach at the current event_seq from GET /runs/{id}.

The correct client behavior, worked out:

1. Track the last event_seq you saw

   Every SSE frame carries an id: — that's your resume cursor. The TypeScript client exposes this as stream.lastEventSeq automatically.

2. If the stream ends without done, ask the run

   Call GET /runs/{id} — the run resource is the source of truth, independent of whatever happened to the transport. Its status tells you what actually happened.

   const run = await client.runs.get(runId);
   if (run.status === "streaming") {
     // still going — reattach
     const resumed = await client.resume(runId, stream.lastEventSeq);
   } else {
     // complete / failed / aborted — read the final message, done
   }

   Copy

3. Reattach with ?after=, or let resume() do it

   The client's resume(runId, afterSeq) wraps runs.stream and automatically falls back to re-reading the message on a replay_window_exceeded response, so this reconciliation dance isn't something you write by hand.

Errors

Every error shares one envelope, extending TurboLLM's existing { error: { code, message } } shape:

{
  "error": {
    "type": "capacity",
    "code": "rate_limited",
    "message": "Too many requests for this tenant. Slow down and retry shortly.",
    "request_id": "req_01J8...",
    "retryable": true,
    "retry_after_ms": 60000
  }
}

type is the coarse class you switch on — frozen at nine values, never extended without a major version:

invalid_request | auth | not_found | conflict | capacity | engine | storage | unsupported | internal

code is the precise, open reason — adding a new one is non-breaking, so don't switch on strings you don't recognize; fall back to handling by type.

Audit log

GET /api/ext/v1/audit is the tenant's own record of every mutation on this surface — who did what, when, and what it returned. It requires the chats:read scope, and like every other read here it's scoped to tenant and owner: one owner can never see another owner's history, even within the same tenant.

curl -s "http://localhost:6996/api/ext/v1/audit?owner=user_4821&limit=5" \
  -H "Authorization: Bearer $TURBOLLM_TENANT_KEY"

{
  "data": [
    {
      "id": "5f1e2a90-...",
      "owner": "user_4821",
      "action": "message.create",
      "target_id": "msg_01J8...",
      "request_id": "req_01J8...",
      "status": 201,
      "key_prefix": "tllm-ext",
      "at": "2026-08-18T00:03:12.441Z"
    }
  ]
}

Only mutations are recorded. action is one of chat.create, chat.update, chat.delete, message.create, message.update, message.delete, run.start, or run.cancel — plus the sentinel request.rate_limited for a request the blanket per-tenant budget refused before any route ran at all. Reads, including GET /audit itself, are never audited: at read-heavy traffic volumes they'd dwarf the mutations without adding any accountability. status is the real HTTP status the caller received, so a refused mutation — a 404, a scope-denied 403, a 409 run_active — is itself part of the trail, not just the successes. key_prefix is the first 8 characters of the presented key: enough to tell keys apart in a log, never enough to reconstruct one.

Filter with ?owner= (defaults the same way every other route does) and ?since=<ISO-8601>. ?limit= defaults to 200 and, unlike every other list endpoint on this surface, isn't clamped to a maximum. Rows come back newest first, and there's no cursor here — for anything larger than one page, set since to the oldest row's at and keep paging backward in time.

Entries are pruned automatically after 30 days (a fixed default today, not yet configurable), on the same 30-second background tick that reaps orphaned runs and expired idempotency keys.

Bring your own database

Storage behind this API is a documented interface — ChatStore — not a fixed backend. TurboLLM ships SqliteChatStore as the default, but any integrator can implement the interface over their own database and configure the daemon to load it:

// config.json
{
  "chatStore": { "kind": "sqlite" }                                    // default
  // { "kind": "module", "specifier": "./my-store.mjs", "options": {} }
}

The interface is deliberately small: 13 required methods — create/get/list/update/delete for chats and messages, plus health() and close() — over a typed core of scope, id, and ordering fields, plus one opaque JSON document for everything else TurboLLM stores today or adds later. Two optional capability groups (branching, folders) and two single-method ones (search, batch) are declared, not assumed — a client can discover what's supported at GET /capabilities before it ever calls a gated endpoint, and an unsupported call answers 501 not_supported rather than failing in a surprising way.

Every adapter — including the built-in SQLite one — is proven against the same conformance suite: scope isolation (adversarially — a query in one tenant never returns another's rows), atomic seq allocation under concurrency, counter maintenance, optimistic-version semantics, blob round-trip fidelity, capability honesty, and error mapping. There's a full worked example — schema, adapter, and a passing run of this exact suite — for Postgres:

Limits

Bounded on purpose — on a single-GPU box, an unbounded queue is a memory leak that presents to your users as a hang.

Check current values live at any time:

curl http://localhost:6996/api/ext/v1/capabilities \
  -H "Authorization: Bearer $TURBOLLM_TENANT_KEY"
# -> { "capabilities": {...}, "limits": { "max_page_size": 200, "max_body_bytes": 1048576, "max_attachments": 4 } }

Exceeding either cap returns 429 rate_limited with a Retry-After header — this surface has one capacity signal today, not a separate per-tenant-vs-whole-box split. A generation that fails after it's already started because the daemon itself is unavailable (GPU contention, an engine hiccup) surfaces as engine_error over SSE instead, not as a direct HTTP status — see the Errors section above.