try-works/role-model

role-model is a protocol for assigning the right model for the right job. Use local and cloud AI together, or route between several cloud providers. - try-works/role-model

9 stars · 1 forks · 0 watchers · TypeScript

README

role-model role-model is an open protocol for capability-aware AI routing, plus a reference router that implements that protocol. It gives a router a durable contract for describing what a request needs, what an endpoint can do, what policy allows, and why a final routing decision was made.

What role-model does Every AI workload eventually faces the same question: which model should handle this request? The answer depends on task type, required capabilities, cost, latency, and whether the model is running locally or in the cloud. role-model makes that decision explicit, explainable, and portable. At a high level, role-model separates AI routing into a few stable pieces:

Requests describe task type, required capabilities, modalities, tool needs, and constraints. Endpoint identities and profiles describe concrete routable endpoints rather than abstract model names. Routing policy applies hard denies, preferences, budgets, and tie-break rules. Observability artifacts record the decision, trace, usage, and observed performance.

The reference router supports hybrid routing across three deployment shapes:

Local / Local - route between local models (e.g., llama-swap peers) based on role, task, and capability Local / Cloud - route between a local model and a cloud provider based on cost, latency, and task difficulty Cloud / Cloud - route between cloud providers (e.g., OpenAI, DeepSeek, Moonshot) based on capability and economics

This means a single runtime can serve a quick chat request from a fast local model, route a complex coding task to a capable cloud model, and fall back to a cheaper cloud endpoint when the primary is degraded, all under one explainable routing contract. Install the runtime For end users, prefer the packaged standalone runtime over a source build. macOS and Linux curl -fsSL https://raw.githubusercontent.com/try-works/role-model/main/scripts/install.sh | sh The installer downloads the latest GitHub Release archive, installs it under ~/.local/share/role-model-router/<version>/<target>/, and exposes a role-model-router launcher in ~/.local/bin. Windows irm https://raw.githubusercontent.com/try-works/role-model/main/scripts/install.ps1 | iex The installer downloads the latest GitHub Release archive, installs it under %LOCALAPPDATA%\Programs\RoleModelRouter\<version>\<target>\, and creates a role-model-router.cmd launcher. Manual downloads If you do not want to use installer scripts, download the matching archive from GitHub Releases.

Platform Archive Launch

Windows role-model-router-win32-x64.zip Role-Model.bat or role-model-runtime.exe

macOS x64 role-model-router-darwin-x64.tar.gz role-model-runtime

macOS arm64 role-model-router-darwin-arm64.tar.gz role-model-runtime

Linux x64 role-model-router-linux-x64.tar.gz role-model-runtime

Installation for Pi The pi-role-model package connects Pi to an externally running Role-Model runtime. Start the Role-Model runtime first, then install the public Pi package: pi install npm:@try-works/pi-role-model For local checkout testing from this repository, install the package directly: pi install ./packages/pi-role-model Inside Pi, run: /role-model setup /role-model doctor /role-model alias list /role-model alias choose /role-model alias use <alias>

By default the package connects to http://127.0.0.1:3456 and registers Role-Model as the role-model provider using /api/role-model/downstream/openai. Set ROLE_MODEL_ENDPOINT before starting Pi to use a different local runtime. Remote endpoints require explicit trusted allowRemote behavior, and runtimes that report authentication.required fail closed unless a future supported token source is configured. For local development installs and the full command reference, see packages/pi-role-model/README.md. Develop from source Prerequisites

Node.js 24 (required for node:sqlite and SEA support) pnpm 10.x (via corepack enable) Go 1.24+ (for llama-swap vendor binary and Windows launcher)

corepack enable corepack pnpm install Smoke test corepack pnpm run smoke For a fuller walkthrough, see docs/public/quickstart.md. Development build Run the bridge and UI in development mode (separate processes):

Terminal 1: bridge server

Terminal 2: UI dev server

Build UI static files

Package the bridge as a single executable

1. Build UI

2. Package bridge SEA runtime

3. Build Go launcher

4. Bundle UI files

Start the bridge server on port 3456 Open Microsoft Edge in app mode (dedicated window) Serve the UI directly from the bridge (no separate dev server)

Documentation

Read this If you want

docs/public/README.md the docs hub

docs/public/introduction.md what role-model is and why it exists

docs/public/quickstart.md a real end-to-end smoke run

docs/public/concepts/how-role-model-works.md the system flow

docs/public/concepts/protocol-overview.md the protocol surface

docs/public/concepts/routing-overview.md how routing decisions happen

protocol/README.md canonical schemas and fixtures

role-model-router/README.md reference router packages and runtime apps

docs/protocol/routing-policy.md routing policy reference

docs/protocol/taxonomy-v1.md taxonomy V1 groups, roles, tasks, and Pi classification

docs/protocol/roles.md role metadata reference

docs/protocol/tasks.md task metadata reference

docs/operations/02-ci-and-release-flow.md CI, release automation, and workflow ownership

CHANGELOG.md release history

Acknowledgements role-model builds on the work of several open-source projects:

llama-swap - the vendored local model lifecycle manager that handles process supervision, request forwarding, and model swapping for local endpoints LiteLLM - the unified LLM API abstraction whose provider catalog, model metadata, and pricing data inform the routing-compatible provider inventory

License This repository is licensed under BUSL-1.1 with a project-specific Additional Use Grant. Internal production use, evaluation, development, modification, and non-production redistribution are permitted under the root license. Hosted or managed third-party services, paid product embedding, and third-party commercialization require a separate commercial license. See LICENSE for the full terms. Contributions require acceptance of the Contributor License Agreement before they can be merged. Only individual contributions are accepted. Please do not submit work owned by an employer, client, company, or other entity unless you personally have the right to contribute it under this project's terms.

source