solvi-ai/solvi

A runtime where decision models propose and your code decides: typed questions, hard rules, quotes checked against the text, escalation with a guaranteed error budget, replayable traces. Works with...

0 stars · 0 forks

README

solvi Decision systems you can check. You describe what to compute with plain Python functions and checks; a model — an LLM, your own classifier or a small local one — proposes where judgement is needed; solvi's checks decide. Every answer comes with a confidence, a reason you can check (the rule's inputs, a formula over computed facts, or a quote with character offsets in the source) and a hash-chained trace that can be re-executed later to confirm the answer or pinpoint the step that was altered. When something cannot be computed or checked, solvi abstains or hands the case to a person instead of guessing; a failed hard check always overrides any model's confidence. solvi 1.0 has two levels. Ready systems you configure: solvi.build (decisions from labelled examples, with a promise on the errors), solvi.Agent (acting in an environment), solvi.Guard (an agent's tool calls) and solvi.Knowledge (what they know, and from whom). Building blocks in solvi.core, which the ready systems are made of, each replaceable by a part of your own. What works but has no measured gain yet is kept apart in solvi.experimental. Install pip install solvi # core: rules, checks, learned answer heads, build / Agent / Guard (numpy, scipy, pydantic) pip install "solvi[model]" # + torch, transformers: ModernBERT field extractors for documents and the decider pip install "solvi[onnx]" # + onnxruntime, tokenizers: the decider (solvi.models.decider) on CPU without torch pip install "solvi[serve]" # + fastapi, uvicorn: `solvi serve app.py:system` — the questions over HTTP (also --mcp) pip install "solvi[mcp]" # + the official MCP SDK for solvi serve --mcp (without it, a built-in stdio server) pip install "solvi[duckdb]" # + duckdb: stored decisions in a DuckDB file (solvi.core.store.DuckDBStorage); [postgres] for PostgreSQL pip install "solvi[lora]" # + torch, transformers, peft: solvi.experimental.lora.adapt_lora, a LoRA adapter per question (experimental) Requires Python 3.11+ (tested on 3.11–3.14). Coming from 0.9: see below. Start Decisions: solvi.build Give the question, labelled examples and the promise — and a slow path if you have one. solvi.build fits System 1, calibrates its guarantee and who answers what it hands over on examples it did not see, stores every decision, and says what it chose: import solvi from solvi.models import llm

s = solvi.build(question, examples, catalog=cat, max_risk=0.02, # examples: [(state, correct answer)] slow=llm(URL, "openai/gpt-oss-120b"), price=(0.037, 0.17), storage="decisions.jsonl") res = s.ask(state) # res.answer, res.by ("s1", "s2" or "human"), res.reasons, res.cost print(s.explain()) # System 1, its signal and promise, who answers each slice, what is not covered On four tasks of the stand (Banking77, Abt-Buy, CUAD, RAGTruth) the defaults matched or beat the hand-written setups with every promise kept on eval, in a quarter to half of the code — closer to the promised level than the hand-written ones, and the slow path was rarely given anything (guide, examples/24_one_entry_point.py). Environments: solvi.Agent and solvi.Knowledge km = solvi.Knowledge("knowledge.jsonl", vocabulary={"here": lambda s, a: s["here"], "wood": lambda s, a: s["wood"]}) km.goal("table", done=lambda s: s["table"]) # an agenda goal: done is code agent = solvi.Agent(env, knowledge=km) # env: reset(seed), actions(state), step(action) → Outcome agent.run(seed=7, steps=150) # explore; the same world again needs fewer slow decisions agent.replay() # every decision re-checked from what it was given solvi.Knowledge keeps what a system learned and from whom: facts with their sources (never the system's own answers), goals and gates, an action model learned from outcomes; a retraction takes back everything derived from the item. solvi.Agent acts on it: System 1 where the knowledge predicts the action works, a search where it does not, gates and predicted refusals as hard checks — protection by default, justified risk (RiskBudget) as an option. In the toy crafting world of examples/25_environment_agent.py the first run takes 61 steps, all decided by System 2, and the same world met again 12 steps, 11 of them by System 1. Growth was shown in environments met again, not on decision streams or a support agent with tools; justified risk lowers the cost of protection without promising parity (docs/agent.md). solvi.build(..., knowledge=km) and solvi.Guard(..., knowledge=km) read the same knowledge. Tool calls: solvi.Guard The agent proposes {"name": tool, "arguments": {...}}; solvi.Guard checks it — the tool is in the catalog, the arguments validate against its types, the values that must come from the conversation are quoted there (and not only from a tool output that says "ignore previous instructions"), your policies (limits, roles, allow-lists) are ordinary hard checks, and an optional decider asks "did the user ask for this?" under act_guard and perturb — then allows it (solvi runs the function), denies it with the reasons, or escalates it to a person. Every decision is a stored, replayable trace; any framework's tool calls go through guard.check / guard.call (guide, examples/19_agent_guard.py). On τ-bench retail no call it let through was refused by the environment, at a cost in solved tasks (the τ-bench row of the task table). Models: solvi.models Types declare questions; a model proposes; checks decide. The fields of a pydantic model are the questions, their types the kinds (one option, several, an ordered score, yes/no); a decider answers them about a text or a JSON / pydantic state with probabilities, a calibrated confidence and act / escalate, and hard checks, constraints and rules still decide. The decider is whichever model you have:

An LLM — solvi.models.llm(base_url, model, api_key=...): any OpenAI-compatible chat-completions server (OpenAI, OpenRouter, vLLM, llama.cpp, Ollama). Nothing to install beyond the core; its JSON replies are validated, and an invalid one escalates, never a guess. A decision service — solvi.models.systemone(url, model): anything that speaks the System One API. A local checkpoint, for offline or cheap cases — solvi.models.decider("solvi-base") (solvi[onnx], no GPU; solvi models pull solvi-ai/solvi-base downloads it). solvi-base is a 150M ModernBERT-base cross-encoder distilled from solvi-large: about 50 ms per question on a CPU (ONNX fp16, 4 threads). Its card is honest about where it stands: 54.5% zero-shot on typed questions over JSON states (the same as solvi-large), 56.3% on Fast Decisions dev — not better than earlier small models there — and 0.602 on the jabr classifier benchmark, where Jev reaches 0.966. A preview: fit it on 30–60 labelled examples of your task and calibrate its escalation on your own stream before relying on it.

import os from typing import Literal from pydantic import BaseModel, Field from solvi import Catalog, Scale, System from solvi.models import llm

class Triage(BaseModel): team: Literal["billing", "technical", "shipping"] = Field(description="Which team should handle this ticket?") urgency: Scale[Literal["low", "medium", "high", "critical"]] = Field(description="How urgent is it?") angry: bool = Field(description="Is the customer angry?") topics: list[Literal["refund", "delay", "bug"]] = Field(description="What does the ticket mention?")

model = llm("https://api.openai.com/v1", "gpt-4o-mini", api_key=os.environ["OPENAI_API_KEY"])

offline: model = solvi.models.decider("solvi-base") — the same questions, the same checks

@cat.check(hard=True, then={"urgency": "critical"}) # a legal threat is critical, whatever the model says def no_legal_threat(ticket) -> bool: return "lawyer" not in str(ticket).lower() questions[1].requires.append("no_legal_threat")

res = System(cat, questions).ask({"ticket": {"subject": "Charged twice", "body": "Refund my double payment!", "customer": {"tier": "pro"}}}) print({q: (r.answer, r.status) for q, r in res.results.items()}) print(res.audit("team")) # the probabilities, the model's fingerprint, what escalated and why A state is read as key paths (customer.tier: pro); an unsure or escalated answer abstains with the reason (system.stats["model_escalated"], ["low_confidence"]). examples/15_typed_decisions.py runs the whole story with a stand-in model, without network. Whatever the model, read what it was measured on before relying on it, and calibrate it on your own labelled stream (part.act_guard, under Building blocks): checks, constraints and escalation are what make the answers safe to act on, not the model alone. The checkpoint contract of a local decider is in docs/decide_format.md; a local checkpoint answers one question per forward pass with the published checkpoints, several with DecideModel.load(..., multi_question=True). Every answer is a value and a confidence, and the types also declare answer primitives: Maybe[T] ("not stated" — solvi.Unknown — is a real answer, unlike an abstention), Span[float] (an exact piece of the text, parsed), Rank[...] (the top k, in order), Estimate[0, 7, 14] (a number with an interval), and evidence quotes on any answer (Claim(value, evidence=[...]), Question(require_evidence=True)) — each checked in the text, from rules or a model (guide, examples/16_primitives.py). Try it

solvi playground: write a decision task in Python and run it, watch the strategist's plan, tamper with a trace and see the replay catch it, learn rules from examples. Its "New in 1.0" tab: solvi.build from labelled examples with explain(), res.checks and then= as a function of facts, quotes matched on a normalized view, the compact journal, the two levels. solvi documents: cited, typed answers from contracts, invoices, receipts, leases and more — the ModernBERT extractor (ONNX) and the decisions both run in your browser; add a field by describing it. solvi arcade: game agents that explain every move — tic-tac-toe, maze, minesweeper, 20 questions, Mafia detective, a bot arena, and "hack the trace". Its "Agent and knowledge (1.0)" tab: solvi.Agent in a small world — run 1 explores (System 2), run 2 in the same world walks what it learned (System 1), the knowledge report with a retraction, and protection vs justified risk (RiskBudget) behind a gate that holds. solvi realms: an endless strategy game whose factions are solvi systems — tested for 100 000 turns: flat decision time (~0.3–0.6 ms), bounded memory and state, every sampled trace replay OK. All run entirely in your browser (Pyodide): no server, no GPU, nothing you type leaves the page. Each Space installs the solvi release it pins (1.0.0). Models: solvi-ai/solvi-large (typed decisions, 396M, preview), solvi-ai/solvi-base (the same answers on a CPU / in ONNX, 150M, preview), solvi-ai/extract-base (fields by description) and solvi-ai/extract-receipts. Each model card states what the model was measured on, how well it does, and its limits; all models are listed at huggingface.co/solvi-ai.

Building blocks (solvi.core) The ready systems are assembled from these; you can use them directly, and put a part of your own at every extension point (a store, a slow path, a strategist, a head, a decider, an action model, ...). docs/building_blocks.md has one reference per protocol, with a conformance check for each. Quickstart (a catalog by hand, no model) from datetime import date from solvi import Answer, Catalog, Question, System

cat = Catalog()

@cat.fn # argument names = facts it reads; function name = fact it sets def days_requested(start, end): return (end - start).days + 1

@cat.fn def remaining_after(balance, days_requested): return balance - days_requested

@cat.check(hard=True, then={"approve": "reject"}) # if this check is False, "approve" is forced to "reject" def enough_balance(remaining_after): return remaining_after >= 0

@cat.check def enough_notice(start, today, days_requested): return days_requested < 5 or (start - today).days >= 14

@cat.rule("approve") def approve(enough_notice): return "approve" if enough_notice else "needs_manager"

system = System(cat, [Question("approve", "Approve the leave?", Answer.choice(["approve", "needs_manager", "reject"]), requires=["enough_balance"])]) res = system.ask({"start": date(2026, 10, 19), "end": date(2026, 10, 23), "today": date(2026, 9, 25), "balance": 14}) print(res["approve"].answer, res["approve"].confidence, res["approve"].why) print(res.state_text()) print(res.trace.replay(system)) Output: approve 1.0 enough_notice = True days_requested = 5 remaining_after = 9 enough_balance = True enough_notice = True {'ok': True, 'steps': 5, 'mismatches': [], 'models': [], 'answers': 'same', 'catalog': 'same'}

With "balance": 3 the hard check fails and the answer is reject with status == "forced", whatever the rule says. solvi.show.show(res, cat) prints answers, the planned flow, the computed state and the replay result in one go. How it works

Catalog. @cat.fn (computation), @cat.check (bool), @cat.extract (value from text, returned as a Quote with offsets) and @cat.rule(question) (answer rule). A part's contract is its signature: argument names are the facts it reads, the function name is the fact it sets. Type hints are optional and become the facts' types (def risk_score(risk_points: dict[str, float]) -> float): producer and consumer types are checked when a part is registered, values are validated / coerced with pydantic at run time, and a value that fails is rejected like an ungrounded quote (safeguard type_rejected). Untyped parts cost nothing. Questions. Question(name, text, Answer.yes_no() | Answer.choice([...]), requires=[...]). Questions without a rule get a small answer head trained from labeled examples (system.fit) or a readable learned rule list (system.learn_rule). Strategist. For each question it walks backwards from the rule's arguments (or the learned features) through the catalog signatures to the keys of init_state, adds the question's required parts (requires) and every check that touches a computed fact. Everything else in the catalog is not executed; the flow records why each part was taken or skipped. Execution. Each part runs once, even if several questions need it. Hard checks and their inputs run first; when one fails, the steps only the settled questions needed are skipped (res.trace.skipped). With System(..., workers=8) independent steps run in parallel threads as soon as their inputs are ready. Results go into the computed state (res.values, res.state_text()) with their provenance; extracted values keep their quote. Each step record is hashed and chained to the previous one in flow order, so the trace does not depend on scheduling. Answers and trace. res[q].answer / .confidence / .why / .status (ok, forced, abstain), plus res.trace.replay(system), which recomputes every step from recorded inputs and reports mismatches, broken hash links and quotes outside the text — and, given the System, a stored answer that is not the one the trace gives.

Grounded decisions. Fuzzy proposes, deterministic decides, everything is in the trace. Each fact and answer records its provenance — given, computed, quoted, decided (a model's choice among options, with probabilities) or learned — and model-backed steps record the model's id and fingerprint, so a replay can tell when the model changed since a decision. A model's quote that is not literally the text at its offsets, or a choice outside its options, is rejected and counted (system.stats); a fallback producer runs or the question abstains. print(res.audit()) shows what each answer rests on, which safeguards fired, and how much of its support is deterministic. A decision without models and one with models are the same system (examples/12_grounded_audit.py). Guarantees, several models, serving

A guaranteed risk. part.act_guard(examples, max_risk=0.10) calibrates on a few hundred labelled examples of your stream so that P(answered alone and wrong) ≤ 10% for inputs like them (conformal risk control) — a share of all inputs, not the error among the answers given alone (calibrate_for(max_error=0.10, method="ltt") bounds that); the audit shows the promise behind every answer, or says there is none. part.conformal(examples) gives the person who takes an escalation a short list of candidates. Near ties escalate (min_margin=), and the answer does not depend on the order the options are listed in (sorted by default). Several models. Any decider — an LLM, a System One service (Jev, Kev, Von, Laya-serve, …), a local checkpoint — combines with the others: Cascade, Vote and Route (solvi.core.deciders.combine) — the next model only when one escalates, an answer only when models of different families agree, or a model picked by code — under one guarantee (examples/20_vote_across_families.py shows a vote with stand-in servers). Serving and operations. solvi serve module:system exposes the questions over HTTP (OpenAPI from the same types), MCP and the System One API; await system.aask(...) runs async parts concurrently with timeouts. TraceStorage keeps decisions with a hash chain across them; solvi diff shows which stored decisions a rule or model change would flip; solvi test, solvi check and the honesty suite (solvi honesty) belong in CI; res.report(format="html") and solvi report decisions.db --html out.html give an auditor one page per decision or per period; store.signature() — 64 bytes kept next to the chain's head — later names the one stored record that was edited and restores its hash. Text in. system.ask_text("please refund order A-10457, 1.5 million RUB, paid on 12 September 2026", textin=TextIn(system, decider, patterns={"order_id": r"A-\d+"})): the decider picks which question the message asks (or escalates when unsure), each input field is read with a quote (found by a deterministic cue finder — chosen over the checkpoint's span pointer by measurement) and a deterministic parser (numbers, dates, enums, yes / no; a date without a year is not guessed), missing required fields are listed for a clarifying question, and the trace says those values were read by a model, not given. solvi serve answers texts at POST /ask_text and as the MCP tool ask_text. Long documents. decider.decision(..., long="retrieve"): a contract longer than the decider reads is split into sections, BM25 picks the few that bear on the question, the decider reads only those, and quotes point into the whole document; the trace lists the sections read. long="full" reads a text whole up to the length a checkpoint trained on long inputs declares (max_len_long), and retrieves within that length beyond it. Corrections. fit heads refit on all kept examples as corrections accumulate; solvi.core.knowledge.memory.attach(part) escalates an answer when similar corrected cases say another one (guide). The audit, show and the safeguard report render in Russian with System(..., lang="ru").

System 1 and System 2 A System under a guarantee is fast, cheap and knows when it is unsure: that is System 1. An LLM, a re-ask loop or a search is slow and costs money per input: System 2. solvi.build and solvi.Agent put them in one system; underneath:

Who answers (solvi.core.dispatch). System 1 is asked first; when its own signals say its answer cannot be given alone — below its guarantee, unlike the calibration examples, an abstention, a broken constraint — the slow path answers, and when that cannot either, or the budget is spent, a person gets the input with both candidates and the reasons. Dispatcher.calibrate(examples, max_ri

source