Portus API & AI Gateway

130/130

Gateway API v1.6.2 conformance tests. No skips.

126k

Requests per second through 3 proxy pods at 256 connections

0.35ms

p99 latency at a fixed 30,000 requests per second

131Mi

Peak proxy memory across the payload benchmarks

Portus 0.2.3 on a 10-vCPU Linux VM on an Apple M4, medians of three interleaved rounds.

Conformance

Every test. Every profile. No skips.

Gateway API v1.6.2, experimental channel. The upstream suite runs in-cluster against a real per-Gateway deployment. Each square below is one passing result.

CoreExtended Conformance details →

Request mirroringPath & host rewriteHeader modificationH2C & WebSocket backendsMethod & query matchingRequest & backend timeoutsHTTPRoute retryCORSListenerSetListener isolationBackendTLSPolicy with SAN validationTLS terminate, passthrough & mixedClient certificate validationTCPRouteUDPRoute

Performance

Fast under load. Light on memory.

Measured with the open gateway-api-bench method: three interleaved rounds on one machine, against the same backend pods as the top-ranked gateway in that benchmark. Every number and every round is published.

Throughput Payloads

Download Upload HTTPS HTTP/2

Portus 0.2.3agentgateway v1.5.0requests per second, bare GET /

p99 at 30,000 req/s

Portus0.35ms

agentgateway0.82 ms

Errors when 1 of 4 backends goes dark

Portus0.025%

agentgateway2.0 %

Peak proxy memory

Portus131Mi

agentgateway383–456 Mi

Method: the howardjohn/gateway-api-bench approach on a 10-vCPU Linux VM on an Apple M4 (k3s, pods at MTU 65485), three interleaved rounds per gateway, fortio in-cluster with one pod per rung, 3 proxy pods each. Medians shown. Numbers from one machine are only comparable with each other. Failover figures come from the gateway-api-bench suite on Docker Desktop with no retry policy; with a `RetryPolicy` Portus returns 0 errors. Every round, including the tests agentgateway wins →

One data plane

Three gateways. One install.

Kubernetes routes, LLM providers and MCP tool servers sit behind the same Gateway, under the same policies, in the same Rust process.

Gateway API AI gateway MCP gateway

The whole Gateway API, compiled straight to protobuf.

• Every route kind. HTTPRoute, GRPCRoute, TLSRoute, TCPRoute, UDPRoute, ListenerSet and BackendTLSPolicy.
• A data plane per Gateway. Each Gateway gets its own Deployment, Service and PodDisruptionBudget, with its address in `status.addresses`.
• Twelve policy CRDs. Timeouts, retries, rate limits, circuit breakers, health checks, CORS, IP allow lists, body limits, Basic and API-key auth.
• No intermediate config language. The controller compiles your CRDs into one protobuf message and streams each Gateway its slice.

gateway.yamlCopy

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: my-gateway
spec:
  gatewayClassName: portus-gateway
  listeners:
  - name: http
    protocol: HTTP
    port: 80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: my-route
spec:
  parentRefs:
  - name: my-gateway
  hostnames: ["app.example.com"]
  rules:
  - backendRefs:
    - name: my-service
      port: 8080

Put every LLM call behind your own keys and budgets.

• Routes on the request body. A streaming JSON scanner reads `model` and `stream` as bytes arrive, then replays them to the provider unchanged.
• Your keys, not theirs. The ledger issues `portus_sk_…` keys and stores only hashes. The provider's key never leaves the cluster.
• Token budgets that hold. Per key, user, tenant or route, reserved before the call and settled after. Every response carries `x-portus-tokens-remaining`.
• Anthropic and OpenAI dialects. Tokens are metered from JSON and streamed responses, and refusals come back in the provider's own error shape.

Claude Code works unchanged

$`ANTHROPIC_BASE_URL=https://llm.example.com ANTHROPIC_API_KEY=portus_sk_… claude`Copy

ai-gateway.yamlCopy

apiVersion: portus-gateway.dev/v1alpha1
kind: AIProvider
metadata: {name: anthropic}
spec:
  kind: anthropic
  url: https://api.anthropic.com
  credential: {secretRef: {name: anthropic-key}}
---
apiVersion: portus-gateway.dev/v1alpha1
kind: AIRoute
metadata: {name: claude}
spec:
  parentRefs: [{name: gateway}]
  hostnames: [llm.example.com]
  requireApiKey: true
  rules:
  - matches:
    - model: {type: Prefix, value: claude-}
    providerRefs: [{name: anthropic}]
---
apiVersion: portus-gateway.dev/v1alpha1
kind: AIUsagePolicy
metadata: {name: daily-cap}
spec:
  targetRef: {group: portus-gateway.dev, kind: AIRoute, name: claude}
  budget: {tokens: 1000000, window: Daily, per: Key}

One endpoint for every tool server your agents use.

Routes on the tool. Match the JSON-RPC `method` and the tool a `tools/call` names, so `github.` lives on one server and the rest elsewhere.

• Federation. Several servers appear as one, with tools named `<server>.<tool>`. No gateway pod holds session state.
• OAuth from your issuer. Bearer tokens from dex or any OIDC issuer are verified locally. Groups become tenants and scopes can gate tools.
• Allow lists and call budgets per key. Refusals arrive as JSON-RPC errors, so clients keep their session.

Add it to Claude Code

$`claude mcp add --transport http github https://mcp.example.com/mcp`Copy

mcp-gateway.yamlCopy

apiVersion: portus-gateway.dev/v1alpha1
kind: AIProvider
metadata: {name: github-mcp}
spec:
  kind: mcp
  url: http://github-mcp.tools.svc.cluster.local:3001
---
apiVersion: portus-gateway.dev/v1alpha1
kind: AIRoute
metadata: {name: mcp}
spec:
  parentRefs: [{name: gateway}]
  hostnames: [mcp.example.com]
  requireApiKey: true
  auth:
    jwt: {issuer: https://dex.example.com, audience: portus}
  rules:
  - matches:
    - method: {type: Exact, value: tools/call}
      tool: {type: Prefix, value: github.}
    providerRefs: [{name: github-mcp}]

Design

Built so the hot path stays hot.

Nothing on the request path.

Keys, budgets and token checks are local lookups against state the ledger pushes in. A request never waits on another service.

Reloads never drop a connection.

Config arrives over mTLS gRPC and swaps atomically. Each data plane acknowledges the fingerprint it applied, and that drives the Gateway's `Programmed` condition.

The network stack is a plug-in.

Routing, policies, TLS and the AI logic live in a stack-independent core. That let us measure two engines side by side and ship the faster one.

Architecture

From CRD to wire in one compile.

Reconcilers feed a shared store, the store compiles to a `CompiledConfig` protobuf, and each data plane receives only its own Gateway's slice.

Kubernetes API Gateway · HTTPRoute · GRPCRoute · TLSRoute TCPRoute · UDPRoute · AIRoute · AIProvider watch portus-controller reconcilers → ConfigStore → CompiledConfig portus-ledger keys · budgets · usage gRPC stream · mTLS snapshots · usage dataplane Gateway A · rama dataplane Gateway B · rama dataplane Gateway C · pingora your services your services your services

Controller

Watches Gateway API and Portus CRDs with kube-rs, provisions a data plane per Gateway, and recompiles on any change. Reconcilers publish events to each other instead of polling.

Data plane

Builds route maps keyed by port and hostname with full Gateway API precedence. Failing endpoints are ejected passively and readmitted with back-off. SIGTERM drains in-flight requests.

Ledger

Opt-in with the AI gateway. Issues keys, keeps budgets as a shared counter synced every second, stores usage, and fetches OAuth issuers' signing keys. It never sits on the request path.

Network stack

We tested two engines. Rama won.

Everything Portus decides lives in a core that knows nothing about the proxy framework underneath. So we ran the same gateway on Rama and on Pingora, on the same machine, and measured. Rama was faster on every payload rung and used 2–2.6× less memory, so it became the default in 0.2.4.

Default since 0.2.4

Rama 0.4

Used unpatched, with Portus's own upstream connection pool. 130/130 conformance, and the AI and MCP gateways run on it.

+3–31 %throughput vs Pingora, every rung

2–2.6×less memory

Default until 0.2.3

Pingora 0.9

Cloudflare's proxy framework carried Portus through its first releases and the benchmarks above. Rama measured faster than it on every payload rung.

Rama against Pingora: a single round on the same machine, 2026-09-15. A three-round comparison ships with the next release's numbers.

Standalone

No cluster? Still Portus.

Point the data plane at one YAML file and it runs as a plain reverse proxy, with the same routing, policies and TLS it runs in Kubernetes.

• ACME built in. Let's Encrypt certificates over `http-01` or `tls-alpn-01`, renewed at two thirds of their lifetime.
• Hot reload. Edit the YAML or renew a certificate and the change applies without a restart or a dropped connection.
• Backends by name. Docker Compose services and VM hostnames, re-resolved every 30 seconds.

PORTUS\_CONFIG\_FILE=portus.yaml portus-dataplaneCopy

acme:
  email: ops@example.com
  cache_dir: /var/lib/portus/acme
  challenge: http-01
listeners:
  - port: 80
    protocol: HTTP
    routes:
      - hosts: ["secure.example.com"]
        redirect: {scheme: https, status_code: 301}
  - port: 443
    protocol: HTTPS
    tls:
      acme:
        domains: ["secure.example.com"]
    routes:
      - hosts: ["secure.example.com"]
        backends:
          - address: "web:8080"   # compose service
            weight: 3
          - address: "10.0.1.2:8080"
            weight: 1

Get started

Two commands to a conformant gateway.

Kubernetes 1.32+ · Helm 3.8+ · amd64 & arm64Copy

# Gateway API CRDs, experimental channel
kubectl apply --server-side --force-conflicts \
  -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.6.2/experimental-install.yaml
Portus: controller, GatewayClass, policy CRDs, mTLS material
helm install portus oci://ghcr.io/portus-gateway/charts/portus-gateway --version 0.2.12 \
  --namespace portus --create-namespace

Star on GitHub Read the docs AI gateway guide