RRSI: Regularized Recursive Self-Improvement of Agent Harnesses
01 · The problem
Evolved harnesses overfit the benchmark they are scored on. RRSI transfers.
RRSI improves every out-of-distribution benchmark without overfitting the split it evolves on. Prior methods do the opposite: large evolve-set gains that shrink or vanish once the benchmark changes, two of them ending below the harness they started from.
(a) Evolve-split gain against out-of-distribution gain, one point per method. RRSI is the only method whose gain grows out of distribution.
(b–d) Held-out scores for the unevolved harness H0, the average prior method and RRSI. RRSI beats the prior average by up to 22.9%.
02 · The idea
Regularize the search, not the harness
Every harness component stays editable. RRSI constrains the loop that edits it: how much one proposal may change, and which measured gains are allowed to stick.
One round of RRSI. The proposer spends a shrinking edit budget and reads the full ledger; the critic screens for leakage before anything is scored; the gate admits a candidate only if it clears the noise floor and pays for its tokens. Schematic; the real rounds are in the explorer below.
Proposal side
how search capacity is spent
Annealed edit budget
Early rounds may bundle a few coordinated edits to find a mechanism; late rounds get one attributable change.
Evidence-aware credit
Every candidate is logged with its hypothesis, diff, score and cost change, so the proposer builds on what worked and stops re-testing what failed.
Structured exploration
When progress stalls inside the noise band, budget is redirected to components the run has never touched.
Selection side
which gains become permanent
Leakage critic
Task names, entities, answers or benchmark-specific logic are rejected before a candidate is ever scored.
Noise-adjusted floor
A gain must clear the variance measured on the unchanged base harness.
Cost rule
Extra inference tokens have to be paid for by measured gain.
Pruning
Components that stop earning their place are flagged for deletion.
03 · Results
Every held-out benchmark improves
Evolve on one suite per domain, then run the harness unchanged everywhere else. Same tools, judge, trials and window as H0; the policy is Claude Opus 4.8.
Pass rate for the coding suites, model-judged rubric scores for Harvey LAB, JobBench and APEX-Agents, win rate against the human expert for GDPval, frozen simulators for EngDesign and Frontier-Eng.
Prior methodsAblationPolicy robustnessUnseen backbone
Cheaper, too. Policy tokens per trial of each final harness against its out-of-distribution average. RRSI is the lightest evolved harness and the only one that clears H0 by more than a point.
WHY IT IS LIGHT
Two rules act on cost directly
The cost rule refuses growth that is not paid for when it is proposed; pruning removes growth that stopped paying for itself since. No prior method carries either.
04 · Evolution explorer
Watch the harness evolve, round by round
Four real runs, every candidate: what it proposed, what the critic said, why the gate kept or dropped it, and the exact diff. Click anything; use ← / → to step.
How the harness adapted, and what the regularizers blocked in between
← prevnext →
Show the final harness diff (H0 → HT)
05 · Citation
BibTeX
copy to clipboard
@article{xia2026rrsi, title={RRSI: Regularized Recursive Self-Improvement of Agent Harnesses}, author={Xia, Peng and Han, Rujun and Wang, Zifeng and Chen, Yanfei and Zhuang, Yufan and Lee, Yoonho and Huang, Chengsong and Yu, Han and CuiZhu, Zhongying and Ming, Yifei and Yao, Huaxiu and Gokturk, Burak and Pfister, Tomas and Lee, Chen-Yu}, journal={arXiv preprint arXiv:2609.24972}, year={2026} }