Pro-Kla/sabrix-bench

⚡ Fast MCP JSON-RPC inspector & multi-turn agent latency benchmark in Rust - Pro-Kla/sabrix-bench

0 stars · 0 forks

README

sabrix-bench ⚡

Ultra-fast, zero-bloat developer CLI and benchmark harness for Model Context Protocol (MCP) traffic inspection and agent loop latency profiling.

🎯 The Problem Developers building autonomous AI agents locally lack visibility into raw Model Context Protocol (MCP) JSON-RPC tool traffic and have no lightweight way to measure per-turn serialization and proxy latency overhead. Legacy approaches introduce massive performance and security taxes:

Legacy Python / Node Proxies: Add 30ms – 50ms of serialization and runtime tax per tool call. SaaS AI Firewalls: Incur 100ms – 250ms of WAN network latency, TLS handshakes, and third-party cloud data egress per turn. Over a 30-turn agent loop, legacy firewalls add 3.6+ seconds of dead wait time and leak raw database queries and internal system commands outside your perimeter.

sabrix-bench gives you real-time visibility into your local MCP tool calls and benchmarks your agent loops in microseconds ($< 2\ \mu\text{s}$) with zero network egress.

🚀 Quickstart & Installation Build & Install from Source

Clone the repository and install the binary

🛠️ CLI Usage & Subcommands 1. sabrix-bench trace — Real-Time MCP Security Inspector Inspect JSON-RPC 2.0 requests (tools/call, resources/read, tools/list) and flag dangerous tool mutations (destructive shell commands, SQL injections, leaked credentials). Run Built-in Demo Scenarios: sabrix-bench trace --demo Inspect from Inline JSON Payload: sabrix-bench trace -p '{ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "database_query", "arguments": { "sql": "DROP TABLE users; --" } } }' Pipe from Stdin: cat mcp_message.json | sabrix-bench trace

2. sabrix-bench bench — Multi-Turn Agent Loop Benchmark Simulate multi-turn autonomous agent loops to measure local parsing overhead, memory serialization, and latency percentiles ($p50$, $p95$, $p99$).

Run a 30-turn benchmark with real-time progress

Benchmark larger context payloads (10x scale)

Output machine-readable JSON for CI/CD pipelines

3. sabrix-bench compare — Architectural Overhead Matrix Output a side-by-side architectural comparison contrasting In-VPC embedded engines against legacy proxies and remote SaaS firewalls. sabrix-bench compare

📊 Benchmark & Latency Tax Breakdown

Layer / Architecture Per-Turn Latency 30-Turn Loop Delay Egress & Privacy Memory Footprint

Sabrix In-VPC Engine < 2 µs (0.002 ms) < 0.06 ms 100% In-VPC (Zero Egress) < 15 MB

Legacy Python / Node Proxy 35.0 ms +1.05 seconds Local Cluster 150 MB – 400 MB

SaaS AI Firewall 120.0 ms +3.60 seconds Full Payload Egress N/A (Cloud SaaS)

🛡️ Built-in Risk Checks sabrix-bench evaluates deterministic security rules locally in sub-microsecond time:

MCP-SEC-001: Destructive Filesystem Operations (rm -rf, mkfs, dd, chmod 777) MCP-SEC-002: Remote Code Execution / Reverse Shells (curl | bash, nc -e, /dev/tcp/) MCP-SEC-003: Destructive SQL Mutations & Injections (DROP TABLE, TRUNCATE, DELETE FROM, WHERE 1=1) MCP-SEC-004-007: Leaked API Keys & Secrets (OpenAI sk-, GitHub ghp_, AWS AKIA, PEM Private Keys) MCP-SEC-008: Sensitive Path Egress (/etc/passwd, ~/.ssh/id_rsa, ~/.aws/credentials, .env) MCP-SEC-009: Unconstrained Arbitrary Execution Tool Invocations

🌟 Deploying to Production? Enforce zero-egress In-VPC MCP security with millisecond-grade deterministic policy control: 👉 Deploy Sabrix In-VPC Gateway

📜 License Licensed under either of Apache License, Version 2.0 or MIT license at your option.

source