Jev

A Jev model doesn't write text. You give it a premise and a hypothesis, and it answers with one label — contradiction, entailment or neutral — and how likely each label is. TurboLLM runs Jev models through vLLM and gives them their own Workspace screen (the Jev Playground). On the API, a Jev model answers structured questions about a piece of content at POST /v1/systemone, and also serves POST /v1/classify and POST /v1/rerank for raw premise/hypothesis work.

What a Jev model is

A Jev model is an NLI cross-encoder (natural language inference). It reads a pair of texts together and decides how they relate. It doesn't chat, it writes no prose, and it isn't an embedding model.

Because the answer is a typed label with a probability for each class, your code can branch on it directly. There's nothing to parse and nothing to prompt-engineer.

Here is a real run. The premise was “A chef is chopping onions in a busy restaurant kitchen.” and the model was asked about three hypotheses:

The first follows, the second is ruled out, and the premise never mentions an apron, so the third comes back neutral. These numbers were measured on 2026-09-19 with OpenJev's qwen3.5-4b-nli-v2 checkpoint served by vLLM 0.29.0 under TurboLLM (WSL2, RTX 5070 Ti), and are rounded to three places.

How TurboLLM recognises a Jev model

TurboLLM reads the model's own config.json. A folder is a Jev model when its first architectures entry ends in ForSequenceClassification and its id2label is exactly three labels that read contradiction, entailment and neutral (any letter case, any order). The labels and the prompt template (nli_template) come from that file. Nothing about them is hardcoded, so a model that lists its classes in a different order still maps correctly. Other classifiers, such as sentiment or toxicity models, aren't Jev models and TurboLLM leaves them alone.

Requirements

• The vLLM engine, active. Install it from the Engines screen (it sets up a multi-GB Python environment). See Run any engine.
• Linux, or Windows through WSL2. vLLM isn't supported on native Windows.
• An NVIDIA GPU. With TurboLLM's defaults (vLLM's GPU-memory setting at 0.92, and the 8,192-token limit below), the 9 GB 4B model showed about 12 GB of a 16 GB card the moment it reported running, and 15.2–15.9 GB, nearly the whole card, from the first requests on (nvidia-smi, RTX 5070 Ti, 2026-09-21). vLLM reserves most of the card for itself, so plan on nothing else using the GPU while a Jev model is loaded. Lower settings and other cards haven't been measured.
• The launch flags are handled for you. For the verified OpenJev checkpoint TurboLLM starts vLLM as a classifier: --runner pooling --convert classify, an architecture override that lets vLLM load the checkpoint's weights, and image and video input switched off. If you add your own extra engine arguments, yours win over the ones TurboLLM would have set.

The only environment this has been run in is vLLM 0.29.0 on WSL2 (Ubuntu 24.04) with a 16 GB RTX 5070 Ti. Native Linux and other GPUs are untested; see What's tested, and what isn't.

Windows: run TurboLLM inside WSL2

vLLM can't run on Windows, so a Windows machine needs a second TurboLLM that lives inside WSL2 and owns the GPU work. The short version:

1. Set up WSL2 with Ubuntu

   In PowerShell: wsl --install -d Ubuntu-24.04. The regular Windows NVIDIA driver is what exposes the GPU inside WSL2; run nvidia-smi in the Ubuntu shell to confirm you can see it.

2. Install Node.js 22 or newer inside Ubuntu

   TurboLLM needs Node 22+, and it has to be a Linux install inside Ubuntu, not the Windows one.

3. Start TurboLLM inside Ubuntu

   npx turbollm, then open http://localhost:6996 in your normal Windows browser. If the Windows TurboLLM is already using port 6996, stop it or start this one on another port with --port.

4. Install vLLM from Engines

   Engines → vLLM → install, then make it the active engine.

There's a full walkthrough on video: Run vLLM on Windows via WSL2 — the real setup.

Get OpenJev

OpenJev (AlexWortega/openjev on Hugging Face, MIT licence) is the model this page was written and tested with. The repo keeps several checkpoints in subfolders. The one that has been tested is qwen3.5-4b-nli-v2: Qwen3.5-4B with a three-label head, about 9 GB of weights.

1. Activate vLLM first

   Discover's search follows the active engine: on llama.cpp it lists GGUF repos only, on MLX it lists MLX repos only. With vLLM active there's no format filter, so searching for openjev can find it. With any other engine active, search won't.

2. Or open the repo by its address

   Paste https://huggingface.co/AlexWortega/openjev into Discover's Import from URL (the link icon beside the search box). It opens the repo's file picker whichever engine is active.

3. Pick the checkpoint and download

   Because the repo holds more than one checkpoint, Discover shows one row per checkpoint, with its size and a Jev model badge. Download the qwen3.5-4b-nli-v2 row. Only that checkpoint's weight, config, tokenizer and template files are fetched, into openjev/qwen3.5-4b-nli-v2/ inside your models folder.

A checkpoint TurboLLM has no verified launch settings for is tagged Not verified rather than hidden. That includes the repo's 35B checkpoint, which is untested. A repo with a single checkpoint downloads exactly as it always did.

Load it

Load the model from Models (or Discover → Load). vLLM must be the active engine.

• A confirmation appears only if something is actively running: a chat reply being generated, a Code turn, or a Routine run. It names what will be interrupted and offers Load anyway. If nothing is running, the model just loads.
• When it's ready you get a toast with an Open Jev Playground button. TurboLLM never navigates for you. Loads you didn't start in this browser (a Routine, an API call) don't toast.
• It takes a chat slot. A Jev model counts against Keep-N models loaded (Settings → Gateway) like any chat model. Loading it from Models or Discover always replaces the model that is running, whatever Keep-N is set to, and interrupts whatever is running on it (TurboLLM asks first if something is). Only a request through the gateway that names the Jev model, such as an API call to /v1/classify with auto-swap on, can load it into a second slot beside a chat model, and only when Keep-N is 2 or more and there is enough VRAM. Either way, Workspace shows only the playground for as long as the Jev model is loaded. On a 16 GB card, at the default setting measured above, there's no room left for anything else on the GPU.

The Jev Playground

While a Jev model is loaded, Workspace shows exactly one thing: the Jev Playground at /workspace/jev. Chat, Code and Routines are hidden, and opening any of their addresses redirects here with a one-line notice. Nothing is deleted: your chats and Code sessions are hidden, not stopped, and everything comes back the moment no Jev model is loaded, at the place you left it.

• Header. The loaded model, its engine and its state (Loading… or running), the labels read from the model's own config, and a Switch model button. Switch model lists your chat models and Jev models. Picking a chat model loads it, which unloads the Jev model, and puts Workspace back to Chat.
• Two editors that are the request. state takes a JSON object or array, or plain text (plain text is sent as a string). questions takes JSON. What you type is what gets sent: there is no form in between. Under each editor a live status line reads Valid JSON or Invalid JSON with the parser's message and where it stopped (the state editor reads Plain text instead, for text that isn't JSON), and a Format button re-indents the text once it parses. A draft that breaks one of the request rules (see POST /v1/systemone) shows its message, with the field path, under the editor, and Run stays disabled.
• Run. The Run button or Ctrl+Enter (Cmd+Enter on macOS). No request is sent until you run: opening the screen, picking an example or editing the text never calls the model. Run is disabled while the model is still loading. While a run is in flight the previous answers stay on screen, dimmed.
• Answers. One card per question, under your own ids. A yes/no question shows its number. A pick-one question shows the chosen option, its confidence and every option's probability. A scale question shows the score with its range, its confidence and every level's probability, in level order. Under the cards the playground says these are NLI entailment scores, not a calibrated decision model, and links to how each number is computed.
• Response and curl. Two tabs under the answers. Response is the raw JSON the endpoint returned. curl is the exact request that ran, ready to copy, so you can move from clicking to code. Opened from a non-local address, the curl adds a comment reminding you to send your API key. The footer shows how long the request took and how many input tokens it used.
• Examples. A dropdown with four inputs: a support ticket with all three question types, yes/no (one question written as a question and one as a statement), routing, and scoring. They load text into both editors only; every answer you see comes from the model you have loaded.
• Your draft is remembered in this browser, so a reload keeps it. Nothing else is: there is no run history and no saved playground.

The playground calls POST /v1/systemone, described below, so what it shows is what your code gets.

What swaps the Jev model out

Other work can take the GPU while a Jev model is loaded, and the two ways it happens behave differently:

• A Routine that pins a different model swaps that model in for its run and then restores the Jev model afterwards. Workspace leaves the playground for the length of the run and then comes back.
• A Code agent or an API client that names another model triggers the gateway's auto-swap (Settings → Gateway → Auto model-swap). That model is loaded and the Jev model is unloaded, and it is not restored. Workspace returns to normal.

A Jev model can't be the model a Routine uses, and it can't be the starting model in onboarding. turbollm launch refuses it (coding agents need a chat model), and the list_models tools mark it kind: jev so agents know not to pick it for chat.

The endpoints

All three live on the same gateway and port as the OpenAI-compatible API. They aren't part of OpenAI's API, so the OpenAI SDKs have no method for them: use plain HTTP.

TurboLLM handles these paths itself rather than passing them through to the engine, and POST /v1/systemone is TurboLLM's own: it is not a proxied engine path. Because POST /v1/rerank is handled here too, it is a Jev endpoint: if you were reaching a llama.cpp reranker on /v1/rerank through the gateway, that path no longer gets there. A request that names a model that isn't a Jev model gets 400 not_a_jev_model. A request with no model gets 400 invalid_request on /v1/classify and /v1/rerank, and 422 invalid_request on /v1/systemone.

Rules every Jev endpoint shares

• model is required and must name a Jev model. Use the id from GET /v1/models, or the model's name, which is its folder name with hyphens and underscores turned into spaces (qwen3.5 4b nli v2). Matching tries the exact id, then the exact name, then the name ignoring case, then a fragment of the name. There's no fallback to whatever is loaded, so chat's local shorthand doesn't apply: an unknown name is a 404 and a non-Jev model is a 400. The one alias, jev-latest, is accepted on /v1/systemone only (see Choosing the model).
• The model is loaded on demand. With Auto model-swap on, a Jev model that isn't running is loaded first, which can unload your chat model, and that first request waits for vLLM to start. With auto-swap off, the request fails with 503 until you load it from Models. The active engine must be vLLM.
• Batching. /v1/classify and /v1/rerank take up to 128 hypotheses or documents per request, sent to the engine as one batch. /v1/systemone takes up to 512 hypotheses per request (see below) and sends them in chunks of 128, one chunk after another. There's no streaming.
• Your text goes in as written. Braces or $ in a premise or document are never treated as template syntax.
• Auth is the same as the rest of /v1. A key is needed only when Require API key is on; send Authorization: Bearer <key> (X-TurboLLM-Auth and x-api-key are also accepted). See API overview.
• Not over Turbo Link. Naming a model on a linked machine returns 400 link_jev_unsupported. None of the Jev endpoints travel over Turbo Link, so call the machine that has the model.

POST /v1/systemone

Ask a set of questions about one piece of content and get one answer per question. You send the content (state) and a map of questions under ids of your own. TurboLLM turns each question into premise/hypothesis pairs, runs them on the Jev model named in model, and turns the model's entailment scores into an answer. There are three kinds of question: yes/no (noul), pick one (choice) and a position on a scale (score). The request and response field names follow TypeSafe AI's public System One API, so a client written for that shape only changes its base URL and key. The numbers behind those fields are worked out by TurboLLM as set out in The mapping, and they are not calibrated.

Request body

Whitespace is never trimmed. JSON parsing puts keys that look like whole numbers first, in numeric order, so if your question ids or option names look like numbers, answers and the tie-break between equal options follow that order.

A question is { "type", "instructions", "criteria" }.

Limits

• 512 hypotheses in total per request. A noul question counts as 1, a choice as its number of options, and a score as its number of levels.
• The body is at most 1,048,576 characters.
• Nesting is at most 32 levels of arrays and objects in any one value. A deeper value is refused with 422 naming the field, because it would otherwise overflow the server's serialiser.

Response body (200)

Numbers are returned as computed, not rounded.

Shape only — the numbers are illustrative. This request has three questions, one of each type. The response below shows the shape of the answers. Its numbers are made up to show the fields, not the output of a real run. They are rounded to three places, and <n> stands for a token count.

curl http://localhost:6996/v1/systemone \
  -H "Content-Type: application/json" \
  -d @- <<'EOF'
{
  "model": "qwen3.5 4b nli v2",
  "state": "I've been unable to connect my payment provider for three days and the integration keeps failing. I'm losing sales, please help as soon as possible.",
  "questions": {
    "urgent": { "type": "noul", "instructions": "Does the message convey urgency?" },
    "team": {
      "type": "choice",
      "instructions": "Which team should handle this message?",
      "criteria": {
        "billing": "Payment, invoices, refunds or subscription charges",
        "technical": "Bugs, outages or integration problems",
        "sales": "Pricing, plans, upgrades or discounts",
        "documentation": "Questions about where to find docs or reference material"
      }
    },
    "mood": {
      "type": "score",
      "instructions": "What is the customer's tone?",
      "criteria": [
        "Calm, just asking or stating facts",
        "Mildly annoyed but polite",
        "Clearly frustrated",
        "Very angry, strong language"
      ]
    }
  }
}
EOF

# Only when Require API key is on, add:
#   -H "Authorization: Bearer <your key>"

import requests

headers = {"Content-Type": "application/json"}
# headers["Authorization"] = "Bearer <your key>"  # only when Require API key is on

resp = requests.post(
    "http://localhost:6996/v1/systemone",
    headers=headers,
    json={
        "model": "qwen3.5 4b nli v2",
        "state": "I've been unable to connect my payment provider for three days and the integration keeps failing. I'm losing sales, please help as soon as possible.",
        "questions": {
            "urgent": {"type": "noul", "instructions": "Does the message convey urgency?"},
            "team": {
                "type": "choice",
                "instructions": "Which team should handle this message?",
                "criteria": {
                    "billing": "Payment, invoices, refunds or subscription charges",
                    "technical": "Bugs, outages or integration problems",
                    "sales": "Pricing, plans, upgrades or discounts",
                    "documentation": "Questions about where to find docs or reference material",
                },
            },
            "mood": {
                "type": "score",
                "instructions": "What is the customer's tone?",
                "criteria": [
                    "Calm, just asking or stating facts",
                    "Mildly annoyed but polite",
                    "Clearly frustrated",
                    "Very angry, strong language",
                ],
            },
        },
    },
)
if not resp.ok:
    raise RuntimeError(resp.json()["error"]["message"])

answers = resp.json()["answers"]
print("urgent", answers["urgent"]["noul"])
print("team", answers["team"]["choice"], answers["team"]["confidence"])
print("mood", answers["mood"]["score"], answers["mood"]["confidence"])

const res = await fetch("http://localhost:6996/v1/systemone", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    // "Authorization": "Bearer <your key>", // only when Require API key is on
  },
  body: JSON.stringify({
    model: "qwen3.5 4b nli v2",
    state: "I've been unable to connect my payment provider for three days and the integration keeps failing. I'm losing sales, please help as soon as possible.",
    questions: {
      urgent: { type: "noul", instructions: "Does the message convey urgency?" },
      team: {
        type: "choice",
        instructions: "Which team should handle this message?",
        criteria: {
          billing: "Payment, invoices, refunds or subscription charges",
          technical: "Bugs, outages or integration problems",
          sales: "Pricing, plans, upgrades or discounts",
          documentation: "Questions about where to find docs or reference material",
        },
      },
      mood: {
        type: "score",
        instructions: "What is the customer's tone?",
        criteria: [
          "Calm, just asking or stating facts",
          "Mildly annoyed but polite",
          "Clearly frustrated",
          "Very angry, strong language",
        ],
      },
    },
  }),
});
if (!res.ok) throw new Error((await res.json()).error.message);

const { answers } = await res.json();
console.log("urgent", answers.urgent.noul);
console.log("team", answers.team.choice, answers.team.confidence);
console.log("mood", answers.mood.score, answers.mood.confidence);

A real response to the request above (OpenJev 4B on vLLM, 2026-09-21). Identical requests differ slightly from run to run; see What the numbers are, and what they aren't:

{
  "model": "<model id>",
  "answers": {
    "urgent": { "type": "noul", "noul": 0.946 },
    "team": {
      "type": "choice",
      "choice": "technical",
      "probabilities": { "billing": 0.309, "technical": 0.505, "sales": 0.18, "documentation": 0.005 },
      "confidence": 0.319
    },
    "mood": {
      "type": "score",
      "score": 2.231,
      "legend": {
        "0": "Calm, just asking or stating facts",
        "1": "Mildly annoyed but polite",
        "2": "Clearly frustrated",
        "3": "Very angry, strong language"
      },
      "probabilities": { "0": 0.026, "1": 0.065, "2": 0.561, "3": 0.348 },
      "confidence": 0.45
    }
  },
  "usage": { "input_tokens": 509, "output_tokens": 0 }
}

The mapping: how a question becomes a number

The premise is the state, and it is the same for every question: a string as written, an object or array as pretty-printed JSON. The question text goes in the hypothesis. Each pair goes through the model's own template (Premise: … Hypothesis: … for OpenJev), and the number TurboLLM keeps from each pair is the model's probability of entailment.

For the example request above, the nine hypotheses (in the order they are sent) are:

urgent  Does the message convey urgency?
team    Which team should handle this message? The correct answer is: billing (Payment, invoices, refunds or subscription charges)
team    Which team should handle this message? The correct answer is: technical (Bugs, outages or integration problems)
team    Which team should handle this message? The correct answer is: sales (Pricing, plans, upgrades or discounts)
team    Which team should handle this message? The correct answer is: documentation (Questions about where to find docs or reference material)
mood    What is the customer's tone? The correct answer is: Calm, just asking or stating facts
mood    What is the customer's tone? The correct answer is: Mildly annoyed but polite
mood    What is the customer's tone? The correct answer is: Clearly frustrated
mood    What is the customer's tone? The correct answer is: Very angry, strong language

The answers are worked out from the entailment probabilities of those hypotheses:

• fit is the strongest raw (un-normalised) entailment among the options or levels, which says whether anything fits at all.
• margin is the gap between the two highest normalised probabilities, which says whether the winner is clearly ahead.
• sd is the spread of the probabilities around score, the standard deviation sqrt(sum of pi × (i − score)2) over the level indices, and n is the number of levels, so the second factor is 1 when all the probability is on one level and 0 when it is split between the two extremes.

Choosing the model

model takes an id or a name, matched exactly as on /v1/classify: the exact id, then the exact name, then the name ignoring case, then a fragment of the name. The alias jev-latest (any letter case, surrounding whitespace ignored) means the Jev model that is loaded; otherwise, among the Jev models whose architecture TurboLLM has verified, the one with the largest size on disk (a tie goes to the one that comes first in your library, which is alphabetical by name); and if none is verified, the largest Jev model in your library. “Verified” is a property of the architecture, not of the individual checkpoint, and “largest” is a heuristic, not a quality guarantee: a bigger checkpoint is the safer default, and a model tagged Not verified is passed over while any verified one exists. When your library holds several checkpoints, pass an explicit model id instead. jev-latest is accepted on /v1/systemone only (not on /v1/classify or /v1/rerank), is not listed in GET /v1/models, and follows the normal auto-swap rules: with auto-swap off, a model that isn't loaded is a 503, never a silent load.

Cost

The whole state is sent with every hypothesis, so the work is about hypotheses × (state tokens + hypothesis tokens), and usage.input_tokens shows what a request came to. A request runs its hypotheses in batches of 128, one batch after another. A request can't be cancelled from the playground; it finishes even if you switch to another screen in the app.

Length

By default a Jev model loads with an 8,192-token limit per request. A state plus one question longer than about 32,768 characters is refused with 422 context_length_exceeded, and the message names state or questions.<id>. If you choose a longer limit in the model's load settings, this character check still refuses over-long input, and anything the engine itself rejects comes back as 400 engine_rejected. Nothing is ever truncated.

What the numbers are, and what they aren't

The numbers come from the model's NLI entailment scores: how strongly a cross-encoder finds that the state supports a statement. They are not calibrated, and this is not a calibrated decision model. A noul of 0.9 means the model found the statement strongly supported, not that it is right nine times in ten. confidence is TurboLLM's own formula (above) and is not a calibrated probability. Compare it across messages for the same question, not across different questions: that is all that was measured.

The accuracy figures on this page were measured through /v1/systemone on 2026-09-21, with OpenJev qwen3.5-4b-nli-v2 on vLLM 0.29.0 (WSL2, RTX 5070 Ti), on one small probe: eight support messages written for it, labelled by one author, run twice with the same result. The comparisons between alternative wordings below were measured earlier with the same hypotheses through /v1/classify. What's tested has the sample sizes.

• A statement reads better than a question. An instruction written as a question works: 0.917 on the probe's 24 yes/no pairs, with instructions only. The same idea written by hand as a statement scored 0.958 on those pairs. If you can, write a statement.
• Write neutral scale instructions. A presupposing instruction skews an ordinal scale upward. On the same eight messages and the same 0–3 scale, “How frustrated does the customer appear?” had a mean error of 0.638 and “What is the customer's tone?” had 0.362, and one billing message scored 1.97 with the first wording against 1.27 with the second.
• Nothing is truncated. An over-long pair is refused (422 context_length_exceeded), never cut down.
• Backticked paths in instructions are not resolved. A path such as ticket.messages[0].text is passed through as text. The model sees it beside the JSON it refers to, because the whole state is sent.
• Not bit-for-bit repeatable. Two identical requests can return probabilities that differ in the second decimal. Across eight identical requests the largest spread we measured was about 0.011 (the yes/no number moved by at most 0.0015), and the values move in small steps, which points at the engine's arithmetic precision. Adding a question moved another question's yes/no number by at most 0.0005 in a test. That is vLLM's batched pooling, not TurboLLM. The hypotheses TurboLLM builds and their order are fixed functions of your request, and the winning option did not change in any repeated run we made, but two options whose scores are within about 0.01 of each other can swap places between runs.

POST /v1/classify

Pairs the premise with each hypothesis using the model's own template and returns a label and probabilities for each pair.

Request body

Response body

curl http://localhost:6996/v1/classify \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.5 4b nli v2",
    "premise": "A chef is chopping onions in a busy restaurant kitchen.",
    "hypotheses": [
      "Someone is preparing food.",
      "The kitchen is empty and silent.",
      "The chef is wearing a blue apron."
    ]
  }'

# Only when Require API key is on, add:
#   -H "Authorization: Bearer <your key>"

import requests

headers = {"Content-Type": "application/json"}
# headers["Authorization"] = "Bearer <your key>"  # only when Require API key is on

resp = requests.post(
    "http://localhost:6996/v1/classify",
    headers=headers,
    json={
        "model": "qwen3.5 4b nli v2",
        "premise": "A chef is chopping onions in a busy restaurant kitchen.",
        "hypotheses": [
            "Someone is preparing food.",
            "The kitchen is empty and silent.",
            "The chef is wearing a blue apron.",
        ],
    },
)
if not resp.ok:
    raise RuntimeError(resp.json()["error"]["message"])

for item in resp.json()["results"]:
    print(item["label"], item["probs"][item["label"]], "-", item["hypothesis"])

const res = await fetch("http://localhost:6996/v1/classify", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    // "Authorization": "Bearer <your key>", // only when Require API key is on
  },
  body: JSON.stringify({
    model: "qwen3.5 4b nli v2",
    premise: "A chef is chopping onions in a busy restaurant kitchen.",
    hypotheses: [
      "Someone is preparing food.",
      "The kitchen is empty and silent.",
      "The chef is wearing a blue apron.",
    ],
  }),
});
if (!res.ok) throw new Error((await res.json()).error.message);

const { results } = await res.json();
for (const r of results) console.log(r.label, r.probs[r.label], "-", r.hypothesis);

The response, using the numbers from the run above (abridged):

{
  "model": "<model id>",
  "results": [
    {
      "hypothesis": "Someone is preparing food.",
      "label": "entailment",
      "probs": { "contradiction": …, "entailment": 0.957, "neutral": … }
    },
    {
      "hypothesis": "The kitchen is empty and silent.",
      "label": "contradiction",
      "probs": { "contradiction": 1.000, "entailment": …, "neutral": … }
    },
    {
      "hypothesis": "The chef is wearing a blue apron.",
      "label": "neutral",
      "probs": { "contradiction": …, "entailment": …, "neutral": 0.998 }
    }
  ],
  "usage": { "prompt_tokens": <n>, "total_tokens": <n> }
}

probs always carries all three labels. Only each row's winning probability was written down from the test run, so the other two are shown as … here rather than invented, and <n> stands for a token count the run didn't record per example. The API returns full-precision numbers; the ones above are rounded to three places.

POST /v1/rerank

Ranks a list of candidates against a query, in the usual Cohere / Jina rerank shape. Internally the query is the premise. Each document is dropped into hypothesis_template to make a hypothesis, the pair goes through the model's own template, and a document's relevance_score is the model's probability of entailment for that pair. The Jev Playground doesn't call this endpoint, so hypothesis_template is API-only.

Request body

Response body

curl http://localhost:6996/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.5 4b nli v2",
    "query": "What is the capital of France?",
    "documents": ["Berlin", "Paris", "Madrid"]
  }'

# Only when Require API key is on, add:
#   -H "Authorization: Bearer <your key>"

import requests

headers = {"Content-Type": "application/json"}
# headers["Authorization"] = "Bearer <your key>"  # only when Require API key is on

resp = requests.post(
    "http://localhost:6996/v1/rerank",
    headers=headers,
    json={
        "model": "qwen3.5 4b nli v2",
        "query": "What is the capital of France?",
        "documents": ["Berlin", "Paris", "Madrid"],
    },
)
if not resp.ok:
    raise RuntimeError(resp.json()["error"]["message"])

for item in resp.json()["results"]:
    print(item["document"]["text"], round(item["relevance_score"], 3))

const res = await fetch("http://localhost:6996/v1/rerank", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    // "Authorization": "Bearer <your key>", // only when Require API key is on
  },
  body: JSON.stringify({
    model: "qwen3.5 4b nli v2",
    query: "What is the capital of France?",
    documents: ["Berlin", "Paris", "Madrid"],
  }),
});
if (!res.ok) throw new Error((await res.json()).error.message);

const { results } = await res.json();
for (const r of results) console.log(r.document.text, r.relevance_score.toFixed(3));

The response, using the numbers measured in the test run:

{
  "model": "<model id>",
  "results": [
    { "index": 1, "document": { "text": "Paris" },  "relevance_score": 0.941, "label": "entailment" },
    { "index": 2, "document": { "text": "Madrid" }, "relevance_score": 0.016, "label": "contradiction" },
    { "index": 0, "document": { "text": "Berlin" }, "relevance_score": 0.008, "label": "contradiction" }
  ],
  "usage": { "prompt_tokens": <n>, "total_tokens": <n> }
}

Paris scored 0.941, Madrid 0.016 and Berlin 0.008, rounded to three places. Add "top_n": 1 to get only the best answer back.

GET /v1/models

Jev models are listed next to your other models and marked "kind": "jev". Use that field to tell them apart. A Jev model gets no claude-… alias, because it can't serve a coding agent.

{
  "object": "list",
  "data": [
    { "id": "qwen3.5 4b nli v2|mlx-<quant>|<bytes>", "object": "model", "owned_by": "turbollm", "kind": "jev" }
  ]
}

The id is the model's key: its lower-case name, a format-and-quantisation tag and its size in bytes. Copy it from your own /v1/models response.

What a Jev model refuses

A Jev model answers only /v1/systemone, /v1/classify and /v1/rerank. Chat, embeddings and Messages requests that would be served by one get a clear 400 that points you at /v1/systemone, and the check happens before any model is loaded, so a mistaken call never swaps a Jev model in.

• POST /v1/chat/completions and POST /v1/embeddings return 400 with code jev_model_wrong_endpoint in the OpenAI error shape. The message reads “‘<name>’ is a Jev model: it labels premise/hypothesis pairs and cannot chat. Call POST /v1/systemone (or /v1/classify, /v1/rerank) instead.” (“cannot produce embeddings” for embeddings).
• POST /v1/messages returns 400 in the Anthropic error shape (invalid_request_error, same message, no code).
• “Would be served by one” includes a request that names the Jev model, and one that names no model (or arrives with auto-swap off) while a Jev model is the loaded one.
• Inside the app, a chat view opened while a Jev model is loaded gets 409 jev_model_loaded. Workspace chat is already hidden, so you'd only see this from a standalone chat address.

Errors

Errors use the OpenAI envelope: { "error": { "message": "…", "type": "…", "code": "…" } }. These are every code /v1/systemone, /v1/classify and /v1/rerank can return. invalid_request is a 400 on /v1/classify and /v1/rerank and a 422 on /v1/systemone.

The refusals for chat, embeddings and Messages (400 jev_model_wrong_endpoint) and for in-app chat (409 jev_model_loaded) are covered in What a Jev model refuses. Trying to start a Jev model on a non-vLLM engine from the app's own load call returns 409 engine_model_mismatch, with the same “Activate a vLLM engine” message.

Use cases

Everything a Jev model does comes down to one question: does this text follow from that text? These are the shapes that question fits. They're patterns, not benchmarks: apart from the kitchen and France examples above and the measurements in What's tested, no results are claimed here, so check any of them against your own data.

What a Jev model is not for: generating or rewriting text, chatting, and producing embeddings. It can only label pairs.

What's tested, and what isn't

If a model like this is already in your library, this release detects it as a Jev model. The case that changes most is one TurboLLM used to list as an embedding model, such as cross-encoder/nli-roberta-base. It's no longer an embedding model: it takes a chat slot instead of an embedding slot, so loading it from Models replaces the model in the main slot rather than loading beside it. It answers /v1/systemone, /v1/classify and /v1/rerank, and POST /v1/embeddings naming it returns 400 jev_model_wrong_endpoint (see What a Jev model refuses). Like every Jev model it needs vLLM, and it stays untested.

Limits

• Runs on vLLM only (Linux or WSL2). There's no native Windows path and no llama.cpp path.
• Three endpoints, no streaming. Classify and rerank take up to 128 hypotheses or documents per request; /v1/systemone takes up to 64 questions and 512 hypotheses.
• The numbers are not calibrated, and confidence is not a calibrated probability. See What the numbers are, and what they aren't.
• Answers are not bit-for-bit repeatable: identical requests can differ by about 0.01 in a probability.
• A /v1/systemone request is not counted as active work, the same as /v1/classify. The model-load confirmation doesn't warn about it, and switching or unloading the model while one runs ends it with an engine_unreachable error.
• The length limit is a default: a Jev model loads with an 8,192-token limit per request. See Length.
• No OpenAI-style chat adapter: a Jev model doesn't chat, and chat requests to it get the 400 above.
• While one is loaded, Workspace is the playground only.
• Not available across Turbo Link.

Troubleshooting

Still stuck? See Troubleshooting for the general fixes, or ask in Discord or GitHub issues.

Credits and naming

OpenJev is the work of its author, AlexWortega, and is released under the MIT licence. TurboLLM is not affiliated with TypeSafe AI, whose own product is also called Jev. In TurboLLM, “Jev” simply names this kind of model.