Stageflow

Open-source runtime for configurable multi-stage agent workflows with typed handoffs, DAG execution, human gates, MCP, CI, and a local operator console.

Author pipeline-owned YAML in your project. Stageflow schedules each stage in a fresh agent session, moves context through explicit envelopes and artifacts, and persists run state under .stageflow/. Pi is the current agent execution backend. Bins: sf and stageflow.

Why Stageflow?

Multi-step agent work breaks down when every handoff is ad hoc — a shell script here, a chat transcript there, no shared contract between steps. You end up re-explaining context, losing artifacts, and unable to run the same flow locally and in CI.

Stageflow treats stages as the unit of composition. You author pipeline-owned YAML — *.pipeline.yaml, *.task.yaml, optional stageflow.yaml manifest — and define whatever workflow fits your domain: release automation, research pipelines, content review, SDLC, ops runbooks, or something entirely custom. Each stage runs in a fresh Pi session, emits a typed envelope for the next stage, and can pause on human-in-the-loop (HITL) gates when you need an operator in the loop.

The same pipeline runs three ways without rewriting anything:

• Locally — sf ui for triage, provider setup, and gate replies
• Headless / CI — sf validate and sf run --json with predictable exit codes
• Via MCP — Streamable HTTP tools when sf ui or sf mcp is running

Stageflow is not an SDLC tool. Software delivery is a popular pattern in fixtures and dogfood flows, but stages are user-authored and domain-agnostic. If you can express a multi-step workflow in YAML, Stageflow can run it on Pi.

Features

• Pipeline-owned YAML — *.pipeline.yaml with inline stages or uses: refs; separate *.task.yaml files; optional repo-root stageflow.yaml manifest
• Path-based CLI — --pipeline and --task take filesystem paths (no bare-id lookup)
• Pi-native — runs on @earendil-works/pi-coding-agent; reuse an existing Pi login (pi_home) or store credentials in ~/.stageflow/agent/auth.json (sf_owned)
• Envelope handoffs — typed stage payloads and artifacts via write_stage_artifact / emit_stage_envelope
• Verified stage execution — pipeline-owned completion checks with durable evidence, automatic repair, and manual recovery (guide)
• HITL gates — operator questions in the console; CI exits 2 when a run is waiting
• Operator console — triage runs, connect providers, answer gates, inspect transcripts at http://127.0.0.1:3847
• MCP endpoint — Streamable HTTP at /mcp when sf ui or sf mcp is running
• CI / headless — sf validate --strict --json, sf run --json with exit codes 0 / 1 / 2
• Parallel stages — pipeline DAG with fan-out and join (see YAML catalog)
• Clonable fan-out — clone one successor N times at completion, then join (see YAML catalog)
• SQLite run store — <git-root>/.stageflow/ state plus per-run workspaces under .stageflow/runs/

Architecture at a glance

The scheduler owns orchestration semantics: DAG readiness, bounded parallelism, fan-out/join, conditional routing, retries, skipped branches, and resumable human gates. Agent execution sits behind AgentPort; Pi supplies the current coding-agent session, while Stageflow owns pipeline state, stage workspaces, handoff validation, and the interfaces used by the CLI, console, MCP, and CI.

See Architecture for component boundaries, execution flow, persistence, recovery behavior, and the tradeoffs behind fresh sessions and explicit handoffs.

Installation

Requires Node.js ≥ 20.

Quick install (macOS / Linux):

curl -fsSL https://raw.githubusercontent.com/tejasghutukade/stageflow/main/install.sh | bash

npm:

npm i -g stageflow
# or
npx stageflow
# or, from a packed tarball
npm i -g ./stageflow-*.tgz

better-sqlite3 ships prebuilds for common platforms. --ignore-scripts is fine when a prebuild exists. Benign node-gyp warnings during install can be ignored if require("better-sqlite3") works.

Harness skills (Cursor, Claude Code, Codex, Pi, OpenCode) — from a consumer project:

npx skills add tejasghutukade/stageflow

Then ask the agent to set up Stageflow. Details: docs/skills-suite.md.

Quick start

In a project directory (preferably a git repo):

sf init

This scaffolds stageflow.yaml, pipelines/hello.pipeline.yaml (inline stage), and tasks/hello.task.yaml.

Run (after connecting a provider — see below):

sf ui                          # operator console at http://127.0.0.1:3847
sf run --pipeline pipelines/hello.pipeline.yaml --task tasks/hello.task.yaml

Expanded walkthrough: docs/quickstart.md

Connect a model provider

Each stage sets a model id in YAML (e.g. anthropic/claude-sonnet-4-5). The matching provider must be authenticated before runs succeed — sf validate does not check auth.

Operator console (easiest for local setup):

sf ui

Open Settings → Providers (or Connect from the rail when nothing is configured) and sign in with API key or OAuth.

CLI (works headless and in CI):

sf providers list
sf providers login anthropic --type api_key
sf providers login anthropic --type api_key --api-key-env ANTHROPIC_API_KEY

Use sf providers list to see provider ids and supported auth types (api_key, oauth).

Credential storage: reuse Pi's shared auth file (pi_home) or keep credentials in Stageflow's global store (sf_owned):

sf providers detect
sf providers source set pi_home    # or sf_owned

Pi is Stageflow's current agent execution backend. Stageflow owns the workflow layer around it, and you do not need Pi CLI /login as a prerequisite when providers are configured through the console or sf providers.

Full reference: docs/providers.md

Operator console

Start the console with sf ui (default http://127.0.0.1:3847).

• Runs — active and recent pipeline runs, capacity, and status at a glance
• Run detail — spatial stage map; select a stage for the gated workspace (logs, files, envelopes, HITL). Created runs show not started with a Start run action until history exists
• HITL reply — answer operator gates (ask_operator) without leaving the browser
• Pipelines — browse manifest-declared pipeline definitions
• Settings → Providers — connect model providers (pi_home or sf_owned credential storage)

Brand assets live under docs/img/ (stageflow-og.svg, stageflow-icon.svg).

Headless / CI

The guest actor is the CLI (sf / stageflow). sf ui and MCP are not required in the job.

sf validate --strict --json

Validate exits 0 or 1 only (no waiting / 2). With no flags, sf validate checks pipelines and tasks in the manifest (plus stages). --pipeline validates that pipeline and its stages only; --task validates that task file. It never proves provider auth or checkout paths.

sf providers login <providerId> --api-key-env <VAR>

If the provider also supports OAuth, pass --type api_key.

sf run --pipeline pipelines/hello.pipeline.yaml --task tasks/hello.task.yaml --json

The process exits 0 when the Run succeeded, 1 when it failed (including a busy start), and 2 when waiting. sf run --json prints one stdout document. ok is true only for succeeded. Busy has no runId.

On a mixed Pipeline, default wait parks the Run (exit 2). --skip-gates fails the Stage (exit 1). A Pipeline with no HITL does not need the flag.

Post-run extraction (dogfooded in Archify PR diagrams):

sf run ... --json --include stages > sf-run.json
sf envelope get --from sf-run.json --stage author-diagrams --format handoff --json
sf skills install --from-zip <url> --skill-name archify

See docs/ci.md for the full CI recipe and .github/actions/sf-run composite action.

State

Runtime state lives in <git-root>/.stageflow/ when inside a git repository (SQLite + per-run workspaces under .stageflow/runs/). Global config and sf_owned auth live under ~/.stageflow/. If .stageflow is missing and .software-factory exists from an older install, the next store open renames it to .stageflow once.

MCP

Host MCP via sf ui or sf mcp at http://127.0.0.1:3847/mcp (URL printed on boot). Sessions are the default. Point a Cursor (or other) MCP client at that URL. Do not run both hosts against the same store.

HITL-aware tools include wait_run, answer_gate, and list_waiting. Full tool list: docs/mcp.md.

Stageflow vs Conductor

Both projects address multi-step agent workflows. They differ in orchestration model and runtime.

If you want deterministic YAML routing across many agents, look at Conductor. If you want stage-bound Pi runs with reviewable envelopes and an operator console for workflows you author, use Stageflow.

Examples

Index: examples/README.md

Documentation

Full docs: tejasghutukade.github.io/stageflow (GitHub Pages — live after merge to main and Pages enabled). Source in docs/.

Develop from source

git clone https://github.com/tejasghutukade/stageflow.git
cd stageflow
npm i
npm run build && npm run ui:build   # ui:build builds the UI and copies assets into dist/ui
sf ui

Run tests: npm test and npm run ui:test. Typecheck: npm run typecheck.

License

MIT © Tejas G

Support: GitHub Issues · SUPPORT.md

Contributing: CONTRIBUTING.md