reharness is a self-improving agent harness: bounded recursive self-improvement in which the agent's own model revises the harness the agent runs in, never the model. A business agent runs requests that repeat. When a request comes back returned, a Revision Session reads the agent's own run of it, changes the agent's Harness Package — its tools, hooks, skills, prompts — and proves the change with a Live Probe: the candidate runs the same request for real. Only a probe that comes back accepted is adopted, and only for that request (Per-Task Adoption). No person picks the change, no model judges it, and the session is never shown an answer: at most accepted or returned per attempt, and which check failed. Everything the session cannot touch is a Frozen Core — the scorer, what the session may be told, the package's contract, the bounds — and package code runs in an isolated worker. What the agent learns from its own runs is harness engineering — a guard, a tool, a prompt — not an answer.

Built and measured on AutomationBench (47 simulated SaaS tools, scored on the world state the agent leaves behind) with Claude Opus 5.5 as both the agent and the session, on a subscription budget. Claude Opus 5.5 scores 42.47% on the AutomationBench leaderboard; with reharness, on the benchmark's held-out finance tasks, it scores 68% at steady state, with no passing task lost; on train tasks a session learned, with no hint in its prompt, to open every unread source before writing and to derive figures by code.

Contents

• Key ideas
• Architecture
• How one improvement happens
• Feedback tiers
• Experiment design
• Results
• Install
• Run
• Repository layout
• Contributing
• Citation
• License

Key ideas

• Per-Task Adoption by an accepted Live Probe. The session's candidate runs the request for real and the verdict that comes back decides; the host applies the rule, no judge model, no person. An accepted task opens no session and its package is never revised; adoption binds to one task, so a change proved on one request reaches no other. Tasks influence each other only through an Experience Library of earlier adoptions.
• A typed, bounded learning signal. What a session may learn is a field of the Sealed Protocol, the feedback tier: the Acceptance Signal alone (accepted or returned, once per attempt), or the signal plus the Rejected Checks' kinds and objects (which message, which row — never a value), plus a Source Inventory of the world's data sources as counts. The grading rows never leave the host.
• The session learns habits, not answers. Given that signal it built the same two mechanisms in independent sessions, with none of the words in its prompt: a guard that holds the first write until every unread source is opened, and a tool that derives a figure from the inputs read. Every calculation tool in an adopted package was written by a session; the seed has none.
• A Frozen Core of four invariants. The Host-Side Scorer, whose grading rows never leave the host; the Feedback Tier, which fixes what crosses to the session; the Package Contract — a fixed shape, byte, count and effort limits, and no answer in the package: a candidate's changed lines are scanned against the train grading rows and a value that belongs to one task is refused; and the Bounds on probes, steps, tokens, time and money, which no session can raise. Around the loop sits the measurement, not the mechanism: a Family Split by task family, protocols sealed and committed before the first test attempt, Integrity Pins on the worker image and the code, a Budget Guard that reserves every model call at its worst case before it is made.

Architecture

The system is S = (W, R, E; K): the Harness Package W a session may change, the Revision Session R that changes it, the Host Transcript E it is judged from, and the Frozen Core K that bounds all three.

The seed carries four generic hooks (a ledger, a triage before the first write, a date anchor, a verify on finish) and one tool that writes nothing, plan_changes. The hook sites are before and after the model call, before and after a tool call, after a turn, and on finish; a hook answers in a fixed shape (block, modify, feedback, pushback), and a guard — a before-tool hook that blocks one call once with its reasons and lets the same call through when sent again — is the cheapest component a session has.

How one improvement happens

A Sealed Protocol names the tasks and how many times each is met (three). The figure follows one task's package through the steps, which are the same for every task; the worked example beneath it is a quarterly tax estimate, drawn from one real trace (finance-v24, effort max).

1. First Pass. Encounter 1 runs the base package. An accepted request is finished: no session, and its package is never revised.
2. Revision Session. A returned request gives R one coding session bound to that task. It reads the Run Digest — each call with its route and error flag, the hooks that acted and what they said, the writes by route, the final text; no score — the Source Inventory (each source read by the agent, read only by the session's own hook, or never read), the Rejected Checks the protocol allows, and the Experience Library.
3. Candidate and admissibility. R edits a copy of the package; the candidate is the task's package plus the diff, taken as the diff rather than as the session's claim. Before any probe the host decides admissibility deterministically, no model asked: within the Package Contract (shape and limits, the manifest against the diff both ways), no literal of one task (the changed lines scanned against the train grading rows), and changed, since the same tree is never probed twice. The package's own tests and a smoke run in the worker sit at the same point as a fail-fast; they decide nothing.
4. Live Probe. The candidate executes the same request once for real. The answer is the new run beside the first, plus the verdict — and, at the items tier, which check that run failed. An unchanged candidate is refused; a changed one may be probed again; four probes per task.
5. Per-Task Adoption. The last candidate whose probe was accepted is adopted for this task; none accepted, the base stays. An accepted probe the session never handed in is declared by the host from the diff and stands. The adopted tree and its lesson go to the Experience Library.
6. Steady State. Encounters 2 and 3 run the task's own package. The report reads encounter 1 (First Pass) against encounter 3 (Steady State), paired by task family with a bootstrap interval.

Tasks run in parallel, a task's encounters in order, and every encounter is on record, so a rerun pays for nothing twice.

Feedback tiers

feedback is a field of the Sealed Protocol, so the number reported with an activity says what the session had. The two tiers measured worse — a judge model deciding from the run alone — are kept for comparison.

Experiment design

• Benchmark. AutomationBench's finance domain: 100 public tasks, each a business workflow over simulated SaaS tools (sheets, mail, chat, accounting systems), scored by assertions on the world state after the run; a task passes when all its assertions hold. A local score on the public set is not a leaderboard score.
• Splits. Tasks are assigned to train and test per task family by hash (50 / 50) and frozen; the split hash is part of K.
• Sealing. A protocol — the tasks, the encounters, the arms, the feedback tier, the base release — is sealed to one file and its reference committed before the first test attempt. A session opens train sessions and the task's own runs, nothing else; the trace store refuses the rest.
• Arms. The learn arm adopts per task; the control arm runs the base every time. The reported number is First Pass against Steady State on the learn arm, paired per task and resampled over families; the base's own flip rate stands beside it (10–17% of tasks flip between two runs of one package).
• Model and cost. The agent and the session are Claude Opus 5.5 through headless Claude Code, every arm at one effort level (medium unless stated; automationbench_v2_max lifts both arms to max). Costs are nominal API-equivalent dollars on a subscription; the Budget Guard reserves every model call at its worst case before it is made.
• What is disclosed with a number. The split, the feedback tier, the effort, the sample size, the paired difference with its interval, and the cost.

Results

Claude Opus 5.5 scores 42.47% on the AutomationBench leaderboard. With reharness, on the benchmark's held-out finance tasks, it scores 68% at steady state. Every reharness number is a sealed final, the protocol committed before the first test attempt; across the finals the harness the loop built gained up to +13 points over the same model without it, and not one passing task was lost.

The leaderboard (v1.0.6) and these runs score different task sets at different settings, and the benchmark promises only directional agreement between them. Costs are for the whole activity, in API-equivalent dollars.

What the loop learned: with the Acceptance Signal alone it corrected a third of the returned tasks and never touched a passing one; with the Rejected Checks and the Source Inventory it built generic mechanisms by itself — open every unread source before writing, derive figures by code — in independent sessions with none of those words in its prompt. The failures that remain are documented: ambiguities in a task's own documents that the signal cannot resolve, a world clock the agent is never told, and the benchmark's noise floor (10–17% of tasks flip between two runs of one package). Full records, every run: docs/RESULTS.md, docs/JOURNAL.md.

Install

Python 3.11 or newer, macOS or Linux (on Windows, WSL); Docker for the worker.

python3 -m venv .venv && . .venv/bin/activate
pip install -e ".[dev,automationbench]"        # installs the `reharness` command
pytest -q                                      # offline: no network, no model
docker build --provenance=false -f docker/worker.Dockerfile -t reharness-worker:py3.12 .

Clone AutomationBench and point [bench] upstream_path at it; examples/automationbench/README.md lists what the adapter imports. The agent and the session run through headless Claude Code on this machine's subscription login (claude auth login); the startup check refuses a login that is not a subscription.

Run

The quick start runs the loop on five train tasks in a new workspace; the first two commands call no model.

reharness bench --config examples/automationbench/quickstart-repeat.toml --phase smoke --checks-only   # $0: splits, scorer contract, input audit, the seed's literals
reharness freeze --config examples/automationbench/quickstart-repeat.toml                              # $0: K, the budget root, the Sealed Protocol; prints the [rsi] lines to add
reharness bench --config examples/automationbench/quickstart-repeat.toml --phase repeat --arm learn    # 5 tasks x 3 encounters, a Revision Session on each returned one: ~$15-25
reharness bench --config examples/automationbench/quickstart-repeat.toml --phase report                # $0

A config names a preset ([harness] preset) and may only tighten it (docs/design/CONTRACTS.md §9, docs/design/EVALUATE.md §6b). finance-v14.toml to finance-v25.toml are the measured activities; their workspaces hold local, git-ignored run records. For another domain, copy the quick start, change [bench] domain, give the copy a new [workspace] root and [rsi] scope_id, and keep [repeat] split = "train" until a test protocol is sealed and committed.

Repository layout

Contributing

Three kinds of contribution are useful: another adapter — a benchmark or a real business tool set — behind the same Package Contract; a feedback-tier experiment with its Sealed Protocol committed before the first test attempt; a reproduction report with config, split hash, effort and cost. Every measured number comes with the disclosures listed under Experiment design. See CONTRIBUTING.md; questions go to Issues.

Citation

@software{reharness2026,
  title   = {reharness: a self-improving agent rewrites its own harness, not its model, and proves each change live},
  author  = {The reharness Authors},
  year    = {2026},
  url     = {https://github.com/e13ven-arch/reharness},
  note    = {Bounded self-improvement of an agent's harness package, measured on AutomationBench}
}

License

MIT, see LICENSE. The Stirrup seed is vendored under its own license in reharness/harness/seeds/stirrup/vendor/stirrup/LICENSE; AutomationBench keeps its upstream license.