Mantis
SELF\_HOSTED: TRUECLOUD\_PROVIDER: AWSAPI\_STYLE: OPENAI
Mantis
A self-hosted LLM gateway for routing, caching, guardrails, and observability across model providers.
01 // UNIFIED\_API
One Interface
A stable chat completions endpoint that sits in front of multiple model targets. Switch providers without changing a single line of client code.
OPENAICLAUDE+MORE
02 // INFRASTRUCTURE
AWS-Native
Built for the AWS ecosystem. Deploys via Terraform to ECS, using ElastiCache for fast caching and Bedrock for foundational models.
ECSCLOUDWATCHBEDROCK
03 // LOGIC\_ENGINE
Policy Driven
Routing, retry, and fallback logic. Define timeouts, cooldown periods, and cache behavior directly in your config.
ROUTINGRETRYCACHING
// Capabilities
01
Configurable Routing
Route requests by metadata, model aliases, weighted targets, and fallback chains.
multi-providerweighted routingfallback chains
Configurable routing: stepped medallion with a pulsing core node
02
Gateway Orchestration
Coordinate validation, cache checks, cooldowns, provider calls, retries, and terminal responses.
lifecycle controlshort circuitretry logic
Gateway orchestration: square wave field in a diamond aperture with one packet on the centre line
03
OpenAI-style API Surface
Send chat completion requests through a single gateway endpoint with optional routing metadata.
streaming supportunified schemasingle endpoint
OpenAI style API surface: split circle with paired registers and a packet on the axis
04
Python SDK
Call Mantis from application code without manually constructing each HTTP request.
pip installableasync clientrouting metadata
Python SDK: call grid with one active cell, routed through by a packet
// Team
Hubert de Parseval Europe
Rey van den Berg Dublin, Ireland
Riz Syed London, UK
Samuel Meka Taipei, Taiwan
// Next Steps