agent-orchestration

A delivery pipeline built from reusable skills that you can point at any codebase.
One task in, one reviewed and QA'd pull request out.

Six pipeline skills, eleven steps, two loops with guardrails, three human gates — plus a
standalone triage skill that feeds production errors back into the tracker. The orchestrator stays
in the main conversation, clarifies requirements with you, and delegates planning,
implementation, review, and QA to separate subagents.

task → plan → branch → implement → review ⇄ fix → QA → fix ⇄ review ⇄ retest → cleanup → report
                                    (×3 max)          (×3 max)

What 20 real runs look like

Most published agent pipelines are a diagram and a promise. This one has outcomes.

The interesting number is the gap between the last two. Review sends work back 10% of the
time; QA finds bugs 40% of the time — on code a reviewer just approved. That ratio is the
entire argument for keeping them as separate steps, and for running QA second.

Here is one of those eight, in full:

The reviewer read every terminal code path and proved a true statement — no path
double-counts session time — and approved. QA then found time being double-counted, via
a race between a countdown timer and a modal dialog that no reading of the diff could
reveal.

Nobody was careless, and the approval was correct. A reviewer reads the change; QA runs
the program and asks what else can happen. Merging the two steps — the obvious
efficiency — deletes the one that catches this.

examples/run-cumulative-stats/ has that run end to end,
with the artifacts the pipeline actually wrote: epic, child issues, PR, review comment, bug
report, fix, re-review, re-test. examples/ also covers the two send-backs —
one where the pipeline caught its own agents writing a deleted symbol back into the
repository, and one that shipped nothing at all.

For what a run costs, the measured numbers are in
examples/run-progress-store-tests/: 1,097,667
subagent tokens across 9 launches, 302 tool calls, 1h 31m wall clock, broken down per step
and per role — plus a costed estimate and an honest argument about whether it was worth it.

Status, stated plainly

Those 20 runs were produced by the ancestor of this repository: one project, one
stack, prompts with the project fused into them. What is published here is a
restructuring — same flow, same guardrails, rewritten role prompts, and a profile
indirection the original did not have.

The restructured version has now been run end to end once, on a real task in a real
repository: examples/run-progress-store-tests/.
Every guardrail behaved as designed, the profile seam held, and the precedence rule caught
three stale facts in the profile on its first outing. That run has measured token counts,
wall clock and a costed estimate — ≈1.7M tokens, ~$10, 1.5 hours for a task a
competent engineer would scope at half a day. Whether that trade is worth it is argued
honestly in the report rather than assumed.

That measured run predates the skill packaging and shared memory migration. It is
evidence for the pipeline, not an end-to-end validation of the new host launchers.

One run is not twenty. Point it at something real in
dry-run first — it touches neither your tracker nor your
remote — and treat your first live run as a shakedown.

Map

• examples/ — what the pipeline produces, and what 20 runs look like
  • run-cumulative-stats/ — one complete run, every
    artifact, including the bug-fix loop
  • run-progress-store-tests/ — the first live
    run of this version: measured tokens, wall clock, cost, and what it found
  • artifacts/ — the output contracts the orchestrator parses
• PIPELINE.md — the runbook: every step, its transition criterion,
  the strict rules
• PIPELINE-GRAPH.md — the same thing as a state diagram
• skills/ — the six pipeline skills and the standalone production-triage
• gates/ — hooks that enforce the owner's merge and release gates
• profiles/ — the one file you write to adopt this
• adapters/ — Claude Code, Codex, generic harness

The idea

The natural way to write agent prompts is to bake the project into them: this reviewer
knows the framework and checks the billing SDK's entitlement flag, this developer knows
the module layout. It works, and it produces six files that must each be rewritten for
the next project and that quietly rot as this one moves.

So the prompts here are split along that seam:

• skills/<role>/SKILL.md — how a reviewer reviews, how a planner decomposes, what a QA
  verdict requires. Stack-neutral, and true regardless of what the project is written
  in.
• profiles/<project>.md — build and test commands, module names, architecture
  invariants, release gates, the stack-specific checklist entries.

The orchestrator resolves the active profile once and passes its path into every
subagent launch. Adopting the pipeline for a new project means writing one profile.
Host adapters control invocation and models; the skill bodies remain shared. Each
stage completes its assignment and returns to the orchestrator without restarting
the pipeline.

Precedence is: observed code > profile > role prompt. Profiles are written by hand
and go stale; roles are told to follow the code when the two disagree and to report the
drift, which surfaces in the final report so the profile can be fixed.

Install in a project

Run everything below from the adopting project's root. Claude Code is shown in full;
the Codex differences are one line each, and the generic harness
needs only steps 1, 2 and 7. The adapter and gate READMEs hold the details.

Upgrading a project that already runs an earlier version (role skills, gate hooks,
a .claude/project-profile.md or role memory from before this library)? Read
Upgrading an existing installation first. Steps 2
and 3 keep anything that already exists under a role's name, so on such a project they
can finish without installing anything new.

1. Vendor the library and write the profile.

git submodule add <this-repo> agent-orchestration
mkdir -p .agents
cp agent-orchestration/profiles/_template.md .agents/profile.md

Fill in every section of .agents/profile.md; example-mobile-app.md
shows the useful level of detail, and the Verification command and Human gates
sections are the ones the rest of this list depends on. The profile lives in your
project, next to .agents/skills/ and .agents/memory/, so it is versioned with the
code it describes and the orchestrator finds it without an explicit path.

2. Link the canonical skills. An existing entry under a role's name is kept and
reported, never overwritten. Every KEPT line is a role that will run its old version:
inspect it and replace it with the link if it is an earlier copy of that role. If
.agents/skills is itself a symlink, read adapters/README.md first.

ROLES="orchestrator task-planner developer code-reviewer qa-tester settings-optimizer production-triage"
mkdir -p .agents/skills
for role in $ROLES; do
  if [ -e ".agents/skills/$role" ] || [ -L ".agents/skills/$role" ]; then
    echo "KEPT .agents/skills/$role -> $(readlink ".agents/skills/$role" || echo 'own directory')"
  else
    ln -s "../../agent-orchestration/skills/$role" ".agents/skills/$role"
  fi
done

3. Install the host launchers. They carry the model and fork settings; the skill
bodies stay in the library. A launcher is what makes a stage run as a separate
subagent: without it the stage runs inside the orchestrator's conversation and review
and QA are no longer independent.

# Claude Code: thin wrappers in .claude/skills/
ROLES="orchestrator task-planner developer code-reviewer qa-tester settings-optimizer production-triage"
if [ -L .claude/skills ]; then
  echo "STOP: .claude/skills is a symlink to $(readlink .claude/skills)."
  echo "Every role would look installed through it. Migrate first: adapters/claude-code/README.md"
else
  mkdir -p .claude/skills
  for role in $ROLES; do
    if [ -e ".claude/skills/$role" ] || [ -L ".claude/skills/$role" ]; then
      echo "KEPT .claude/skills/$role"
    else
      cp -R "agent-orchestration/adapters/claude-code/skills/$role" ".claude/skills/$role"
    fi
  done
  grep -L '^context: fork' .claude/skills/*/SKILL.md 2>/dev/null \
    | grep -E "/($(echo $ROLES | tr ' ' '|'))/" | sed 's/^/NOT A FORK: /'
fi

The last command lists the installed roles whose launcher does not fork; only
orchestrator should appear. A project that links .claude/skills to
.agents/skills, a common earlier layout, stops here until the
migration replaces the
link with a real directory.

Codex: copy agent-orchestration/adapters/codex/agents/<role>.toml into .codex/agents/
for the five stage roles instead; see adapters/codex/README.md.

4. Install the gate hooks and their configuration.

mkdir -p .claude/hooks
cp agent-orchestration/gates/hooks/*.py .claude/hooks/
cp -n agent-orchestration/gates/hooks/gates.example.json .claude/hooks/gates.json

Edit .claude/hooks/gates.json from the profile's Human gates section: protected
branches, the files agents may not open, every command that publishes a release. With
no release rules the release gate blocks nothing. Codex: the same files go to
.codex/hooks/. Copying over older hooks of the same name replaces their logic; port
any project rules they hard-coded first (see upgrading).

5. Run the self-test before wiring anything. A broken PreToolUse hook blocks every
later tool call, so this step is not optional.

python3 .claude/hooks/selftest.py        # must end with "0 failed"

6. Wire the hooks into the host. Merge the hooks block from
adapters/claude-code/hooks/settings.example.json
into the tracked .claude/settings.json. Codex: merge
adapters/codex/hooks.example.json into
.codex/hooks.json, then review and trust the hooks with /hooks; an untrusted hook is
silently skipped.

7. Install the git pre-push hook. It sees the real push from any tool and is the
second layer behind the command parsers.

mkdir -p .githooks
cp agent-orchestration/gates/git/pre-push .githooks/pre-push
chmod +x .githooks/pre-push
git config core.hooksPath .githooks              # the owner runs this, once per clone
# git config --add agent-gates.protectedBranch develop   # if the base branch is not main/master

Run that git config yourself: merge-gate.py blocks every change to core.hooksPath
so that an agent cannot switch the hook off later. Where the hosting plan allows it, add
a server-side branch rule as well; a local hook cannot stop a merge made through the web
UI with your credentials.

8. Commit the installation. .gitmodules and the submodule pointer,
.agents/profile.md, the .agents/skills/* links, .claude/skills/*,
.claude/hooks/* including gates.json, .claude/settings.json and
.githooks/pre-push. Leave .claude/settings.local.json untracked, and let
.agents/memory/ appear through a pipeline run rather than an empty directory.

9. Check that the gates are live, then do a dry-run.

echo '{"tool_name":"Bash","tool_input":{"command":"git push origin main"},"cwd":"'"$PWD"'"}' \
  | python3 .claude/hooks/merge-gate.py; echo "exit $?"     # expect "Blocked by merge-gate" and exit 2

Then, in Claude, run /orchestrator Task: <a small real task>. Mode: dry-run. It touches
neither your tracker nor your remote (see Dry-run). Confirm that each stage
ran as a separate subagent rather than inline, that the planner read your profile, and
that the final report names the verification command. Record what is actually enforced
in the profile's Enforcement installed field; the pipeline reports gaps from there.

Upgrading an existing installation

A project running an earlier version of this pipeline, or the ancestor it came from,
needs four things beyond the checklist. Each one fails silently if skipped.

• Skills. Replace earlier copies of the seven roles in .agents/skills/ with links
  to the library, and follow the Claude adapter's
  migration when
  .claude/skills is a symlink. Older roles under other names (for example
  ai-orchestrator, ios-developer) stay discoverable next to the new ones; retire
  them once the new commands work, so nobody invokes the wrong orchestrator.
• Profile. Move .claude/project-profile.md or wherever the old profile lives to
  .agents/profile.md, update Repository root, and add the sections the
  template requires that it lacks. Earlier profiles usually
  miss Verification command, Human gates, Review exclusions and
  Production error source.
• Memory. The new roles read .agents/memory/<role>/ under their new names. Notes
  kept under old role names are invisible to them until merged as described in
  memory/README.md.
• Gate rules. Earlier hooks often hard-code project rules that gates.json now
  holds: protected branches, secret names, upload tools. Port them before step 4
  overwrites the scripts. A project-specific check the config cannot express, such as
  allowing an export unless its options plist uploads, stays as an extra hook beside
  the shared ones rather than inside them. An existing release-push checker becomes the
  allow_base_push script; it is called with three arguments
  <remote_sha> <local_sha> <branch>, so a checker that insists on two needs a short
  wrapper, or every release push is rejected.

Codex discovers the canonical packages through per-role links in the adopting project's
.agents/skills/. Claude uses thin skill launchers in .claude/skills/ to supply its
model and fork settings. Install only these roles and preserve unrelated skills.

In Claude, invoke /orchestrator <task>; in Codex, invoke $orchestrator with the task.
The orchestrator stays in your conversation while each stage runs separately. You can
also invoke a stage skill directly when you want only that stage. Skills respect the
scope and mode you supplied; selecting a stage does not start the full pipeline.

The roles

Rules worth knowing before you run it

A handful of these exist because the obvious behaviour is the wrong one:

Every code change made after QA begins passes review before QA can be called green.
QA bug fixes are code. Skipping the review on them is how an unreviewed change ships
under a green verdict.

You hold three gates; agents hold everything between them. Accepting a design
decision, merging the PR, and publishing a release are yours. The rest runs without
you. Instructions alone are advisory, so the merge and release gates are also enforced
by hooks (gates/) — an agent that tries to merge is stopped, not trusted.

One command defines "done". The profile names a verification command — build,
tests, cheap static checks, a short summary. The developer runs it before reporting,
QA runs its fullest variant, and the reviewer doesn't re-report what it enforces.

Review checks the change against what was asked. A compliance pass compares the
PR with its decision records and issue criteria, both ways: missing work and work
nobody asked for. Findings are Important (blocking) or Nits (at most five, never
blocking). Bug fixes start from a failing test committed before the fix.

Loops cap at three iterations, then stop and escalate. A review loop that hasn't
converged in three passes is telling you the task is under-specified or the finding is
wrong. A fourth pass doesn't discover that.

Bugs found outside the current change are filed but never block it. QA labels them
pre-existing; Step 6 skips them and Step 8 refuses to close them. Filing everything and
scheduling separately beats the alternatives — silently not filing, or blocking a
finished PR on unrelated debt.

Unmerged work is not closed. Issues get a PR reference comment and stay open until
merge is confirmed. Closing the wrong one erases a tracked task.

No blanket git add .. Only durable artifacts, path by path. Anything uncertain goes
in the report for a human to decide.

Step 0 resolves missing product decisions before planning. The orchestrator can
clarify them with you in the main conversation and record the agreed design artifacts.
Small operational tasks can proceed without design documents. This keeps the stages
working from the same decisions.

Harness support

Each Codex role TOML includes commented model and model_reasoning_effort examples.
Uncomment them in the installed .codex/agents/<role>.toml to configure that role;
otherwise host defaults apply. See Codex model configuration.

Every adapter is a pointer to skills/<role>/SKILL.md. Runtime files explain
launching and model selection; PIPELINE.md remains the shared workflow. All hosts
use .agents/memory/<role>/ in the adopting project and explicitly load the
shared memory rules.

Existing agents/<role>.md paths remain compatibility pointers for earlier loaders
and historical references. For existing installations, follow the adapter's migration
instructions to switch orchestration into the main conversation and retire old Claude
agent registrations. In particular, do not copy Claude launchers through a
.claude/skills symlink into the canonical skill directory.

Also here

• conventions/ — swappable repo conventions the roles reference:
  the issue tracker (GitHub gh by default), triage labels, and where domain docs live
• memory/ — the agent memory layout, and the rule that keeps it from
  becoming a second, stale copy of the profile

Dry-run

The whole pipeline with no writes to the tracker or the remote — plan to markdown, local
branch, no PR, bugs as a list. Useful for evaluating it on a new project before pointing
it at anything real. See the table at the end of PIPELINE.md.