Make every LLM call swappable.
Gargi Reflex · autonomous model caching for LLM calls
Make every LLM call swappable.
You write the prompt. Reflex learns from your LLM's answers and serves the repeat calls locally, in milliseconds.
You do the prompt design. Reflex does the data science.
$pip install gargi-reflexcopy
REFLEXcard\_arrival5 ms
REFLEXexchange\_rate5 ms
HOLDOUTcompromised\_card3,600 ms
REFLEXtop\_up\_failed5 ms
LLMwhy\_verify\_identity · low confidence3,600 ms
Served locallyHoldout agreement
95%teachershadow → assist
Banking77 end-to-end demo · stub teacher · laptop CPU · 13,082 calls
One minute, from 3.6 s to 5 ms.
Banking77 end-to-end demo · stub teacher · laptop CPU
What Reflex does
1. 01
It watches.
Your typed LLM calls (routing, intent, moderation, extraction) go to your model exactly as today. Reflex logs each input and answer. 2. 02It proves.
It trains a small model on CPU in minutes, then tests it on data it never saw. Unless agreement, calibration and coverage all pass, nothing is swapped. 3. 03It swaps, and keeps checking.
Confident calls are served locally in about 5 ms at $0. Everything else, plus a permanent holdout slice, still goes to your LLM, which takes back over if agreement drops.Built for Python teams whose product asks an LLM the same kind of question thousands of times a day.
89.0%
of calls servable at 95% agreement
Banking77 · Gemini 3.8 Flash teacher · bge-base · untouched test split · CI 85.2–92.8%
95.1%
agreement with a stronger LLM it never saw
Gemini 3.1 Pro · on the calls the student serves
3.6 s → 5 ms
median latency, LLM vs local
Vertex AI Gemini Flash, n=30 · MiniLM on laptop CPU, 4.7 ms p50
\~$51k/yr
projected at 100k calls/day
Projection · Banking77-like decision · $1.66 per 1k calls · 84.5% served locally
THE CONTRACT
If anything in Gargi fails (no model, low confidence, a model that won’t load, a corrupt database, a bug), the call goes to your function, exactly once. Your exceptions propagate unchanged.
Tested by injecting faults at 19 points of the request path, in every lifecycle state.
Get started
$pip install gargi-reflexcopy
Reproduce our numbers in 90 seconds:
$ python -m examples.banking77.run\_demo
Everything about Reflex →Docs →GitHub ↗
Also from Gargi Labs
Coding harness & Meta harness →In developmentIndic Language SLMs →Research preview · Gargi-M1 live