Run decision models locally.

Run decision models locally.

Ask typed questions about any text or JSON and get calibrated answers in milliseconds. Private, open source, on your own hardware.

DownloadBrowse models

ollaya · zsh

$

ollaya run decider --preset agent '{

"request": "Fix the typo in README.md",

"command": "git push --force origin main"

}'

| Answers returned by the model| Question | Answer | Probability | | ---------------------------------------- | ------------------------------ | ----------- | | action | block | 0.53 | | on\_task | no | 0.75 | | risk | 1.23 / 2 could lose local work | 0.15 | | destructive | yes | 0.90 |

$

Real output: decider:2b answered in 178 ms on an RTX 4090.

Fast

Decisions in milliseconds.

A decision model answers in a single forward pass, with no token-by-token generation. On your own GPU, a five-question request to Laya takes about 10 ms, end to end through the HTTP API.

8–10ms

Laya on Ollaya

RTX 4090, five questions, end to end

236–276ms

TypeSafe Jev

Hosted API, median request

Every model, one scale · median latency, lower is better

• laya:multilingual8.1 ms
• laya:en9.6 ms
• gliclass14.7 ms
• nli20.4 ms
• decider:0.8b155 ms
• decider:2b190 ms
• TypeSafe Jevhosted API236–276 ms
0100200300 ms

Ollaya: median of a five-question request through the HTTP API on an NVIDIA RTX 4090 (laya in fp16, the others in fp32). Jev: median request latency of the hosted API in third-party benchmarks (AbdelStark/jev-benchmarks, nibzard/decision-model-benchmark), which includes the network. Setups differ, so read it as an order-of-magnitude comparison.

Drop-in compatible

Speaks TypeSafe's API.

Ollaya serves `/v1/systemone` and `/v1/models` with TypeSafe's request and response shapes. The official TypeSafe Python SDK 0.7.1 works unchanged against a local server.

Request

# Point the TypeSafe SDK at Ollaya
export TYPESAFE_BASE_URL=http://localhost:11435
export TYPESAFE_API_KEY=local        # any value works
export TYPESAFE_DEFAULT_MODEL=laya
…or call the compatible endpoint directly
curl http://localhost:11435/v1/systemone -d '{
    "model": "laya",
    "state": "Can I get an invoice for last month?",
    "questions": {
      "intent": {
        "type": "choice",
        "instructions": "What does the customer want?",
        "criteria": {
          "invoice": "Needs an invoice or receipt",
          "refund": "Wants money back",
          "other": "Anything else"
        }
      }
    }
  }'

Response

{
  "model": "laya:en",
  "answers": {
    "intent": {
      "type": "choice",
      "choice": "invoice",
      "confidence": 0.9547,
      "probabilities": {
        "invoice": 0.9698,
        "refund": 0.0172,
        "other": 0.013
      }
    }
  },
  "usage": {
    "input_tokens": 43,
    "output_tokens": 0
  }
}

TypeSafe compatibility guide

Open models

Open weights, ready to pull.

Pick by what you need: laya is the fastest, decider the most accurate, von reads up to 8,192 tokens, and qwen3guard screens text for safety. The models page shows each one’s accuracy and speed.

• layaOpen decision models from Convai Innovations. Typed, calibrated answers to choice, score and yes/no questions in a single forward pass, in English and 100+ languages.322m · 421m
• deciderDecoder decision models by Mapika on Qwen3.5: the answer is read from option-letter logits in one forward pass. decider:2b scores 0.591 on typed decisions.0.75b · 1.9b
• nliZero-shot classifiers by Moritz Laurer: every option becomes a hypothesis scored for entailment. The most accurate encoder model on typed decisions in our tests.396m · 435m
• gliclassInstruction-following zero-shot classifier by Knowledgator: all options of a question are scored in one pass, so cost barely grows with the number of options.439m
• qwen3guardSafety guard by the Qwen team: is a text safe, controversial or unsafe, and which unsafe category? It answers its own built-in questions, in 119 languages, in one forward pass.0.6b
• decisionDecision models by the vLLM Semantic Router contributors: a fully fine-tuned Qwen3.5 backbone plus an endpoint head that scores every option at its own last token against the question, in one forward pass per question. 16k-token rows.0.75b
• kevDecision models by Jared Palmer: a LoRA on a Qwen3.5 base plus a pointer head that scores every option at its own span, in one forward pass per question. Calibrated with Kev's own temperature.0.76b · 4.2b · 7.9b
• vonDecision model by Victor Hugo Panisa on ModernBERT-large: every option is scored at its own marker, all options of a question in one pass, with an input-conditioned calibration. 8k-token context.395m

Browse all models

Your data stays yours

Private by default.

Tickets, emails and user messages are often the most sensitive data you have. With Ollaya they are scored where they already live.

• ### Local
Runs on your machine with ONNX Runtime, on the CPU or an NVIDIA GPU. The server listens on 127.0.0.1 by default.
• ### Open weights
Weights come from their authors’ Hugging Face repositories, pinned to a commit and checked against sha256. Ollaya never re-hosts them, and the runtime is Apache-2.0.
• ### No per-token fees
Run as many decisions as your hardware can handle. No metering and no API bill.
• ### Calibrated

Platforms

Runs where you work.

A desktop app and a command line for macOS, Windows and Linux, and a Docker image for servers. Every model runs on the CPU; an NVIDIA GPU on Linux, Windows, WSL 2 or Docker takes a request down to milliseconds.

| macOSApple silicon, macOS 14+ | Desktop appMenu bar app.dmg | Command lineInstall script | GPUApple GPULaya and NLI on MLX | | ----------------------------- | ------------------------------------------ | ----------------------------------------- | ------------------------------------ | | Windows10 and 11, x64 | Desktop appDesktop app.exe or .msi | Command linePowerShell script | GPUNVIDIA, CUDA 13Command line | | Linuxx86-64 | Desktop appDesktop appAppImage, .deb, .rpm | Command lineInstall scriptsystemd service | GPUNVIDIA, CUDA 13 | | LinuxARM64 | Desktop appNot available | Command lineInstall scriptsystemd service | GPUCPU only | | WSL 2Linux on Windows | Desktop appNot available | Command lineInstall scriptSame as Linux | GPUNVIDIA, CUDA 13 | | Dockeramd64 and arm64 | Desktop appNot available | Command lineImage on GHCR | GPUNVIDIA, CUDA 13:cuda image, amd64 |

Install for your platform

NVIDIA GPUs need driver R580 or newer; the install scripts fetch the CUDA libraries only when they find one. On a Mac, laya and nli run on the Apple GPU through MLX; other models, AMD and Intel GPUs, and the Windows and Linux desktop apps use the CPU.

Get up and running in minutes.

One binary, one command: `ollaya run laya`.

Download

macOS, Windows, Linux and Docker · Apache-2.0 · GitHub