bulyaki/Credence
Portable local probabilistic inference runtime. Contribute to bulyaki/Credence development by creating an account on GitHub.
0 stars · 0 forks
README
Credence Credence is an embeddable local inference runtime for typed probabilistic decisions from GGUF language models. It is built on a pinned llama.cpp dependency and is intentionally independent of any agent framework. A decision is scored from the model's next-token distribution over a fixed set of permitted labels. No text is generated, so a call costs one prompt decode and returns a boolean, the probability the model assigned to true, and the uncertainty diagnostics needed to know whether that probability is worth anything. Status Phase 1 is in progress. The repository has a real model-backed boolean decision path, explicit profile-bound labels with stated semantics, a Qwen3 ChatML-style profile, a generic raw-text profile that resolves labels on BPE vocabularies, single- and multi-token scoring, top-token diagnostics, two labelled evaluation sets, and Platt (temperature plus bias) calibration fitted from them. The public C ABI, multi-choice operations, daemon, and MCP adapter are later phases; see the roadmap below. Build The canonical local configurations are CMake presets: cmake --preset mac-cpu-golden cmake --build --preset mac-cpu-golden ctest --preset mac-cpu-golden mac-metal-release is the fast configuration on Apple Silicon. Every build preset has a matching test preset. The test suite includes an end-to-end CLI check that is skipped, not failed, until the smoke model below is fetched. Models Model files are never committed. tools/fetch-test-models.py owns the pinned manifest and verifies SHA-256 on every fetch:
Name File Size Role
qwen3-0.6b Qwen3-0.6B-Q4_0.gguf 0.4 GB plumbing smoke tests only
qwen3-4b Qwen3-4B-Q4_K_M.gguf 2.5 GB demos, evaluation, and calibration
python3 tools/fetch-test-models.py # smoke model python3 tools/fetch-test-models.py --name qwen3-4b # demo model The 0.6B model exercises every code path but is too small to make reliable decisions: on the checked-in 22-case evaluation it answers true for almost everything. The 4B model gets 21 of 22 raw and 22 of 22 after calibration. See docs/phase-1-progress.md for the numbers. Usage Probe a model: build/mac-cpu-golden/credence probe --model tests/data/models/Qwen3-4B-Q4_K_M.gguf Make a boolean decision: build/mac-cpu-golden/credence decide \ --model tests/data/models/Qwen3-4B-Q4_K_M.gguf \ --profile profiles/qwen3/qwen3-decision-v2.json \ --context "Invoice #4471 was issued on 3 March. The full balance remains unpaid." \ --proposition "The invoice has been paid." The result is JSON with value, raw_probability (always the probability of true), calibrated_probability (null until a calibration is supplied), entropy, top_two_margin, answer_conformity, the competing top_tokens, and the model and profile identifiers the numbers belong to. Pass --untrusted-evidence to wrap the context in the profile's untrusted delimiters; the result reports evidence_marked_untrusted so callers can see that the flag took effect. Without --profile, the built-in generic profile renders a plain instruction prompt ending in Answer: and resolves A/B labels, falling back to a leading-space encoding when the vocabulary merges the label into the prefix. Evaluation and calibration tools/evaluate.py runs a labelled JSONL set through the CLI and reports accuracy, mean negative log-likelihood, Brier score, and the minimum answer conformity, so a formatting failure is visible separately from a decision failure: python3 tools/evaluate.py \ --credence build/mac-cpu-golden/credence \ --model tests/data/models/Qwen3-4B-Q4_K_M.gguf \ --profile profiles/qwen3/qwen3-decision-v2.json \ --evaluation tests/data/evaluation/boolean-smoke.jsonl \ tests/data/evaluation/boolean-business.jsonl \ --verbose tools/calibrate.py fits a calibration profile on the same data: python3 tools/calibrate.py \ --credence build/mac-cpu-golden/credence \ --model tests/data/models/Qwen3-4B-Q4_K_M.gguf \ --profile profiles/qwen3/qwen3-decision-v2.json \ --evaluation tests/data/evaluation/boolean-smoke.jsonl \ tests/data/evaluation/boolean-business.jsonl \ --output calibration.json The profile is Platt scaling, sigmoid(logit(p) / temperature + bias), fitted with Platt's smoothed targets so a set the model already separates still has a finite optimum. Pass --calibration calibration.json to credence decide; value then follows the calibrated probability and the output says so in decision_basis. The fitter warns when the set is small. The checked-in sets are 22 cases: enough to demonstrate the mechanism, not enough to trust the resulting numbers in production. See docs/probability-semantics.md for what each probability field does and does not mean. Native packages Credence ships as a self-contained native package per operating system rather than a single universal format: ./build.sh On Apple Silicon this builds the Metal package; on Intel macOS it selects the CPU-golden preset. The archive is written to dist/ as credence-v<version>-macos-<arch>.tar.gz. On Windows, run build.bat for the CPU package, or set CREDENCE_WINDOWS_VARIANT=cuda before running it for the CUDA package. The result is dist/credence-v<version>-windows-x64-<cpu|cuda>.zip. Roadmap
Boolean decisions, profiles, scoring diagnostics, calibration (current). choose, score, and rank operations over explicit candidate sets. Model and profile fingerprints enforced on every calibration and result. Stable C ABI and libcredence for embedding. Resident daemon so the model is loaded once, plus an MCP adapter. Language bindings and family profiles for Llama and Gemma models.
License Copyright 2026 Sonus Immersivus. Credence is released under the Apache License 2.0. You are free to use, modify, and redistribute it, commercially or otherwise, as long as you keep the copyright notice and the attribution in NOTICE. llama.cpp is bundled under its own MIT license; see THIRD_PARTY_NOTICES.md.