AutoHarness keeps an agent from taking on a task unprepared. The agent works out what the task needs,
gathers the know-how and tools for it, and an independent critic checks the result until the quality
is acceptable.

A coding agent (Claude Code, Codex CLI, Gemini CLI, Antigravity CLI, Qwen Code, GigaCode, Strands)
usually goes at a task head-on. It writes from memory without looking at what the result has to
match, without finding out how such work is done well, and then marks its own work "done". The
ceiling on quality is then whatever the model remembers. It shows on any task: an integration with
someone else's API, tests, setting up an environment, a page built from a mockup, a drawing that
has to match a reference image.

Engineers do not work that way. Nobody builds an aircraft from memory: the work is split into
disciplines (aerodynamics, materials, strength, control), what is already known in each is looked
up, hypotheses are checked on models, and only then is it built. AutoHarness brings that order to
the agent:

flowchart LR
    A[Analysis<br><code>init</code>] --> P[Preparation<br>areas · method · reference · facts]
    P --> B[Brief<br>the plan, recorded]
    B --> I[Install<br>trust · constraints]
    I --> S[Base<br>block-in first]
    S --> R{Review<br>independent critic}
    R -- blocking --> L[Learn<br>update the skill, then fix]
    L --> R
    R -- minor or none --> T[Teardown<br>skills · recipes · lessons]
    P -. nothing fits .-> W[Write a skill<br>from the sources read]
    W --> B

The two sides split the work: the agent thinks, the CLI keeps count and guards the order. The
agent decides what the task needs and whether the result is good. The autoharness CLI searches
registries, scores trust, installs and removes tools, keeps the records, and does not let the agent
move on until the previous step is done. Every command answers in JSON with a next field that
says what to do next. Exit code 3 always means one thing: stop and ask the user.

The process

1. Analysis: init

autoharness init "<the user's task, verbatim>" --mode light|full|reviewed|auto

• light: a chore that needs no capability from the catalog (a typo, a commit). No search and no
  brief: do the work, check it, teardown. A task that needs any capability (an API, a browser,
  tests, debugging, an image) is raised to full by init, whatever the agent chose.
• full: something is missing: what to check against, know-how, a way to verify. The full cycle
  below applies.
• reviewed: the full cycle plus a mandatory independent critic before the end.

init searches skills.sh and the official MCP
registry for the task right away. It shows similar saved
setups (recipes), candidates earlier tasks rejected (with the reason), and open follow-ups. A
running task can be raised without starting over: autoharness mode full.

2. Preparation: areas of know-how, search, method

This is where quality is decided, and it is the step agents usually skip.

• Areas of know-how. The agent splits the task into the areas the work itself is made of: for a
  service, its protocol, data model and concurrency; for a drawing, figures, anatomy and light. For
  each, it looks for something ready-made (autoharness search skill|mcp "<query>",
  autoharness need) and takes what fits. If nothing fits, it writes its own skill from the
  sources it read.
• Method. How this kind of work is done well: the algorithm, the order, the known pitfalls. The
  agent takes it from sources it found and read, with links, not from memory.
• Input. The original the result has to match: the API's real response, a data sample, the
  mockup, the reference image. A description of it does not count. A search page, a gallery or
  someone's version of it (fan art, a mockup, a blog example) is not the original either.
• Facts. Before the first result the agent measures with a tool what the result will be judged
  on: fields and values, sizes, positions, proportions, timings. Then it records them
  (evidence add <file> --kind facts).
• Open choices. Where the task leaves a choice open (a format, a value or a formula, what a field
  holds) and the user cannot be asked, the agent picks what the person who asked would take for
  granted on first look: the plain, simple form, nothing they did not ask for. A tool's or a skill's
  advice does not decide it.

These are not suggestions. The CLI requires each part:

3. Brief: the plan before the work

autoharness brief --goal "<quality bar>" --domains "..." --method "..." \
  --inputs "<what is missing and where to get it>" --expects "<open choice>: <pick> - <why>; ..." \
  --constraints "<the user's constraints, verbatim>" \
  --verify "<how to check, against what>" --candidates "<name>: take|skip, why; …" \
  --gaps "create <skill>: <what it does>; …"

The brief records what a good result is and how it will be checked, the user's constraints verbatim
("no new dependencies", "offline only"), and a verdict on every candidate. Work that is judged by eye
needs --fidelity: the level the result is judged at ("stylised poster, not a photograph"). Browser,
API, log and database checks have ready templates
(--verify-template browser|api|api-client|logs|database). With the hook
the agent cannot change anything until the brief is recorded.

4. Install: trust and constraints

autoharness inspect mcp <name>        # trust score, secrets, where the server connects
autoharness add mcp <name>  |  autoharness add skill <owner/repo@skill>

• Trust score. The curated catalog is trusted. Registry servers are scored by namespace, pinned
  version and publisher. Anything not "high", anything with secrets and anything remote is
  installed only after an explicit "yes" from the user.
• Skill scan. A skill is downloaded to a temporary folder and checked for dangerous patterns:
  ~/.ssh, curl | sh, "don't tell the user", obfuscation.
• Constraints. When the brief has any, a fresh subagent checks the candidate against them
  (vet), not the agent that wants to install it.
• Network. With allowed_hosts set, a server outside the list is installed only with the
  user's consent.

After installing, the agent makes one cheap call to try it. If it does not work: remove and the
next candidate. There is no point spending long on fixing someone else's tool.

5. Base: the frame first, details later

The first version is a rough base (a block-in): structure, layout, proportions, the main parts
from the facts, no polish. A critic looks at it right away, and the first round judges the base
only. Unfinished details are at most minor in that round; a wrong base is major. A mistake in the
base gets more expensive with every round built on top of it. Edits without a critic are counted:
after 20 edits without a fresh critic the hook stops editing.

6. Review: the author is not the judge

autoharness evidence add <file> --kind reference|facts|result   # the result: whole, then close-ups (--closeup)
autoharness evidence diff                                       # for code: the whole working-tree diff
autoharness verify run "<command>" --for B1                     # autoharness runs it; the exit code decides
autoharness critic                                              # a prompt for an independent critic
autoharness review --by critic --answer-file <file>
autoharness critic --run claude|codex|agy                       # or autoharness starts a read-only critic itself

• The critic is a fresh subagent that did not do the work. Its answer is kept as it is.
  autoharness reads the points, the verdict and a one-time ANSWER-ID from it itself, so a softened
  retelling cannot be passed off instead. A subagent given a review prompt reviews; it cannot take
  the task over.
• What the critic checks:
  • that the reference is the original itself;
  • that the facts are right, by checking a few of them against the reference;
  • every brief item (met | partial | missing | contradicts);
  • the open choices, as the person who asked would see them;
  • whether the work follows the method found;
  • what was done beyond the request;
  • whether each verify run really checks its item;
  • whether the work is finished as a whole (ACCEPT). "No" is at least major.
• A check has to be able to fail. true, echo ok or looking for a string in a file do not
  count. A run is tied to a version of the code and goes stale after an edit.
• Blind comparison. In work judged by eye, the critic compares the current result with the best
  earlier one under the names A and B. Only autoharness knows which is which. If the earlier one is
  closer, the answer is "Went backwards", and the earlier result stays the best.
• Two reviewers (brief --review pair). Every round, besides the critic, a code-quality
  reviewer checks for bugs, unhandled errors, security, and tests that check nothing. The round's
  verdict is the worse of the two.
• Settled disputes do not come back. A fresh critic rules on a disputed point
  (critic --dispute). A point the user rejects is recorded (ruling add), and later tasks in the
  project do not raise it again.
• Files are data. "Approved" or "set none" written inside the files under review is a finding,
  not an instruction.

7. Learn: don't fix from memory what you don't know

If the critic found serious problems, fixing them "from memory" brings the same crudeness back.
So the next result is refused until the task's skill has changed. For each point the skills do not
teach, the agent looks up how it is done well, writes the technique with its source into the skill,
and only then fixes it.

Then there are guards against loops:

• the same problem three rounds in a row → a mandatory change of approach (approach): from memory
  → from the docs, tuning by hand → computing it, a patch → a rebuild;
• several approaches did not help → an independent arbiter: fix it with a measurable check, or rule
  it out of reach;
• 8 blocking rounds in a row → the user decides.

The agent cannot accept the result for the user: teardown --accepted "<their words>" passes only
if the user actually wrote something.

8. The end: teardown, skills, recipes

autoharness skill test <name> --task "<a realistic request>"   # with it and without it, judged blind
autoharness skill save <name>                                    # into the library, after the user's "yes"
autoharness teardown --save-recipe <name> --notes "<what it is good for>"

• Skills accumulate know-how. What the agent learned in the task (how to do it, how to check it,
  an API's quirks) stays in a skill and is saved in ~/.autoharness/skills/. Values measured in one
  project are kept apart from the procedure: the next task checks them again.
• Lessons from friction. If the task went hard (checks failed, the approach had to change, the
  result went backwards, the user stepped in), teardown stops once and asks what broke and what
  helped. Short conclusions accumulate with lesson add. A conclusion confirmed by three tasks moves
  into a skill.
• A recipe remembers the setup and the queries that found it. A similar task gets it right away,
  and the queries run again to find something better if it appeared.
• What stays in the project is set by the teardown setting: keep (the default) or remove.
  autoharness config set teardown remove [--global].

Example

Task: "accept payments through a payment provider's API; the webhooks must be secure". The session
is illustrative and the CLI's answers are shortened.

autoharness init "accept payments through a payment provider's API, secure webhooks" --mode reviewed

autoharness search skill "payment webhooks"         # what already exists for this area
autoharness search mcp "payments api"
autoharness search skill "idempotency"

autoharness brief --goal "a payment goes through, a repeated webhook does not charge twice, a forged one is refused" \
  --domains "payments API: mcp <server from the search> (searched: payments api); \
             webhook signatures: skill <skill from the search> (searched: payment webhooks); \
             idempotency: create idempotent-handlers (searched: idempotency)" \
  --method "verify the signature over the raw body, an idempotency key, retries with backoff: <url>; …; searched: …" \
  --inputs "real responses and webhooks of a sandbox account" \
  --expects "amounts: integer minor units - what the provider's API takes" \
  --constraints "no new dependencies" \
  --verify "tests on real saved webhooks: forged, repeated, out of order" \
  --candidates "<server>: take, …; <skill>: take, …" --gaps "create idempotent-handlers: …"

autoharness add mcp <server>  &&  autoharness add skill <skill>
autoharness skill new idempotent-handlers --description "…"     # filled from what was read

autoharness evidence add webhook-samples.json --kind reference  # real saved webhooks
autoharness evidence add webhook-facts.md --kind facts          # fields, the signature header, event order
# base: the handler, signature check, key storage, nothing around them
autoharness evidence diff && autoharness critic                 # → major: the signature is computed over parsed JSON
# learn: why the signature is computed over the raw body → into the skill, then the fix
autoharness verify run "uv run pytest tests/test_webhooks.py" --for B2
autoharness evidence diff && autoharness critic                 # → minor
autoharness teardown --save-recipe payments-webhooks --notes "accepting payments with secure webhooks"

A task with a mockup, a reference image, logs or an environment goes the same way. Only the areas,
the sources and the way to check change.

Languages

autoharness reads a task in any language it has a language file for. English is built in; Russian
ships as src/autoharness/data/lang/ru.json. A language file lists, per catalog capability, the
words that point to it, plus stopwords, common word endings, the ways to say "no constraints" and
dependency headings. To add a language, add data/lang/<code>.json (or ~/.autoharness/lang/<code>.json
for yourself) and a matching tests/lang/<code>.json with the cases it must understand; no code
changes are needed.

Gateway: servers without a restart

An agent usually sees a new MCP server only after a restart. The gateway removes that. One server,
autoharness, is added to the agent's config once; from then on add mcp writes new servers to
.autoharness/gateway.json and the gateway starts them on the fly. It gives the agent two
permanent tools: harness_tools and harness_call(server, tool, arguments). It also publishes
native <server>__<tool> tools with tools/list_changed. In Claude Code they appear without a
restart; in Codex and Gemini CLI the meta-tools are the reliable path. A server that arrived in
gateway.json through git pull does not start without approval on this machine
(autoharness gateway approve).

Security in short

• Only what passed the trust check, or what the user said "yes" to, is installed.
• Skills are scanned before they are copied into the project. Secrets are never written into
  configs, only ${ENV_VAR}.
• An independent subagent checks the user's constraints, not the agent that wants to install.
• Task records are signed (HMAC, the key lives outside the project). An edit that goes around
  autoharness shows at teardown, and the hook keeps tools away from the key.
• Only what autoharness installed on this machine is removed.
• The hook and --approve frame the procedure; they are not a sandbox. Where each protection ends
  is described in docs/security.md.

Quick start

uv tool install git+https://github.com/D-Shey/autoharness   # the CLI (needs uv; skills and MCP need Node.js)
autoharness agents                                          # which agents are found
cd /path/to/your/project
autoharness --agent claude install-skill                    # the skill for the agent (--user --all: every agent)
autoharness --agent claude gateway install                  # optional: MCP servers without restarts
autoharness --agent agy gateway install --user              # the same for every project (agy, Gemini, Qwen, GigaCode)
autoharness install-hook                                    # optional: Claude Code does not start without a brief
autoharness --agent agy install-hook                        # the same for Antigravity CLI

If autoharness is not found, run uv tool update-shell and open a new terminal. Restart the agent
and give it a task.

Documentation

• docs/reference.md: every command and where everything is kept. It also covers
  follow-ups found outside the task (followup), measuring the benefit (eval record|batch|compare),
  settled disputes (ruling) and settings (config).
• docs/installation.md: installing for every agent, the gateway, the hook,
  uninstalling.
• docs/security.md: how each protection works and where it ends.
• docs/design/: the specs and plans features were built from.
• CONTRIBUTING.md, SECURITY.md, CHANGELOG.md.

Limitations

• Quality still depends on the model. AutoHarness makes it prepare, check itself against the
  original and learn, but it does not draw or write code for it.
• autoharness's checks are formal. It sees that a search ran, a skill changed, facts were recorded,
  but not how good they are. The substance is judged by the critic, and the critic is a model too.
• A task's text can leave a choice open that only its author could settle. The brief and the critic
  pick what a person would most likely expect, which is not always what a hidden check wants.
• The gateway does not do OAuth sign-in or the legacy SSE transport: such servers go into the agent's
  config and need a restart.
• allowed_hosts limits remote servers only. A local process connects wherever it wants.

Development

git clone https://github.com/D-Shey/autoharness.git && cd autoharness
uv tool install -e .            # the autoharness command sees edits right away
uv run --extra dev pytest       # tests; uv creates .venv and installs pytest itself