9lives by QualityMax

An open-source testing framework for coding agents, built by QualityMax.
Run your Playwright tests, connect your agent through MCP, and inspect the
assertions and execution receipts behind each result. Extend the framework to
Android with experimental Appium support. Start locally without an account or
use 9lives in QualityMax Cloud.

Quickstart ·
Runnable first example ·
Interactive demo ·
Website · Memes · Documentation ·
npm SDK ·
CLI releases

If 9lives is useful to you, star it on GitHub
or run 9l star.

Run your existing Playwright tests

Install the native CLI on macOS or Linux:

curl -fsSL https://9lives.run/install.sh | sh
export PATH="$HOME/.local/share/9lives-runner/bin:$PATH"
9l plan tests/checkout.spec.ts --format json
9l run tests/checkout.spec.ts --timeout 2m --attempts 1 --format json

Run from your project root and replace the path with an existing spec. Follow the
quickstart for prerequisites, expected results and
troubleshooting, or try the synthetic fault example.

From CLI v0.4.8, run 9l doctor from the application project before the
first scan. It checks
local prerequisites and gives repair commands without downloading browsers or
loading project config. Use 9l doctor --sdk for SDK runs. See the
setup diagnostics guide for limits and first-run steps.

From 0.5.4, the CLI includes 9l update (ask and install),
9l update --check (check only), and 9l update --yes (explicit scripted
approval). See updating the CLI.

Your project supplies its Playwright dependencies, browsers, configuration, and
assertions. The SDK is optional. Native v0.6.0 binaries support macOS, Linux and
Windows on x64 and ARM64. See Windows ZIP installation
and the existing-project guide. Chromium is qualified;
other browser engines are unqualified.

Connect the tools to a coding agent through native MCP. For Claude Code:

claude mcp add 9lives -- 9l mcp

The agent setup guide covers Codex, Claude, and the optional
QA skill. Native tools expose execution, assessment,
healing, page inspection, reproduction generation, and fix confirmation.
qmax-code is the separate QA terminal
that can run standalone or connect to QualityMax.

Inspect a real experiment

The demo leads with an application fix: the same reviewed test fails before
the patch and passes after it. Experimental fix confirmation compares the two
Git revisions. Three injected request faults then each cause an assertion failure.
Inspect the patch, scroll the console, and download the evidence extracts.

The last experiment records seven Chromium executions on one synthetic profile
app, captured on 13 October 2026. Opening the demo does not execute tests or call
an agent. It works in light and dark themes and as a single offline HTML file.
To execute the eight-chapter lab yourself, follow the release lab guide.
The earlier Demo Bank locator repair is also reproducible.

From a failure to a checked fix

1. Reproduce: run a reviewed Playwright spec with an assertion that fails on the broken app.
2. Review: inspect your agent's proposed application patch or locator repair; preserve the intended behavior.
3. Confirm: experimental 9l confirm / MCP confirm_finding runs the same reproduction on each Git revision in isolation.
4. Challenge: experimental 9l prove injects supported fetch/XHR faults and reruns the test.

A passing receipt means the assertions passed and the execution report was
validated. Confirmation shows that the test distinguishes the revisions. A
caught fault means an assertion failed under that fault. None establishes full
requirement coverage. See the confirmation and fault testing contracts.

Local first, with a cloud workflow

Run 9lives on your machine or CI without a platform account. For hosted test
management, execution, and team workflows, use QualityMax Cloud.
QualityMax coding-agent sessions run a qualified, pinned 9lives version and read
its receipts alongside QualityMax-managed test results. A local pass and a
cloud-verified result are reported separately. See the
integration contract for version and evidence boundaries.

What is available

• Local execution: plan, run, status, result, and cancel bound execution and record attributable terminal receipts. Each failed test carries a class (assertion, timeout, action timeout, environment), receipts count flaky passes and, with the SDK, the assertion steps the worker reported. run --wait-for waits for the application first, so one that never comes up leaves the run incomplete instead of failing its tests.
• Assessment: 9l assess advises on waits, skips, assertions, requirement mappings, and source provenance.
• Page observations: 9l scan runs nine check categories; 9l inspect ranks controls and locators.
• Optional SDK: named steps and bounded goals alongside explicit Playwright assertions. Goal completion is unverified.
• Experimental: supported locator healing, revision confirmation, and fetch/XHR fault testing. The demo qualifies one synthetic app in Chromium.
• Android (experimental): Appium and UiAutomator2 execution on the documented emulators and phone; see the Android driver guide.
• Planned: independent behavioral verification, verified replay, iOS, and broader Android device qualification.

Add the optional SDK

Install the published SDK in your Playwright project:

npm install --save-dev @9l/[email protected] @playwright/[email protected]
npm exec playwright install chromium

Use Node 24 for the qualified runtime (Node 22 minimum) and Playwright 1.61.1
through 1.64.1; an existing project can keep its version in that range. On Linux, use
npm exec playwright install --with-deps chromium for browser dependencies.

Install the Go CLI separately. With Go 1.27.3 or newer:

go install github.com/Quality-Max/9lives-runner/cmd/9l@latest
9l --version

Ensure Go's binary directory is on your PATH. Prebuilt CLI releases target
macOS, Linux and Windows on amd64 and arm64, so their users do not need Go. See
the download and checksum instructions for availability and
platform selection. Windows ZIPs start with v0.2.0; earlier releases have
macOS/Linux assets only.
The npm package supplies the fixtures; it does not install
the CLI or browsers.

Run with SDK steps

Keep your application's existing Playwright configuration and reviewed
assertions. For an application serving a checkout page:

import {test, expect} from '@9l/playwright';

test('checkout confirms an order', async ({page, n9l}) => {
  await page.goto('http://localhost:3000/checkout');
  await n9l.step('place the order', async () => {
    await page.getByRole('button', {name: 'Place order'}).click();
    await expect(page.getByRole('status')).toHaveText('Order confirmed');
  });
});

Start the application, then run from the test project:

9l run tests/checkout.spec.ts --sdk --format json

For a complete runnable example that needs no application server or model,
clone this repository and follow the local checkout demo.
For an existing application, see the integration guide.

What the result establishes

The runner records a terminal outcome for every required test and validates
SDK evidence against the owning run, job and attempt. Required failures,
missing evidence, unexplained skips and interruption prevent an overall green
result. Receipts persist under .9lives/receipts/; inspect them with
9l result <run-id> --format json.

A passing execution means the test's assertions passed and its report was
validated. It does not prove that the assertions fully cover the requirements.
Independent behavioral verification remains planned.

Playwright's own verdict is not the evidence. Qualifying its modifiers for 9l
found two upstream defects, reported as
microsoft/playwright#43676
and microsoft/playwright#43738:
on 1.61.1 through 1.64.1 a test.skip() in an after hook reports a test that
ran and passed as skipped, and a test.fail() there turns a failed test into
an expected failure, so 1 passed and exit code 0 can describe a failing
run. With --sdk, the receipt reports runtimeExpectedFailures and
skippedAfterAssertions for results relabeled while the test ran, and
9l assess flags test.fail() in an after hook. See
results relabeled while the test ran.

Check the page you changed

Before a coding agent calls a web change done, give it evidence from the page
itself. 9l scan loads the page once in Chromium with the project's own
Playwright and reports graded findings with reproduction steps and suggested
fixes:

9l scan https://your-app.dev --format markdown
9l scan http://localhost:3000/ --allow-private-network --format json --out scan.json
9l scan http://localhost:3000/ --allow-private-network --baseline scan.json --fail-on-new
9l inspect http://localhost:3000/login --allow-private-network

The Markdown report opens with the grade and the categories to fix first:

Grade: 🔴 F (0 / 100) · 15 issues found

Category	Issues	Worst
Console errors	██████████ 4	🔴 high
Accessibility	████████░░ 3	🔴 high
Security headers	██████████ 4	🟠 medium

--baseline compares against an earlier JSON result and reports new, fixed
and unchanged findings, so a pipeline can fail only on regressions.
9l inspect returns the page's ranked locators (test id, role and label
before text and position), each control's stability and a testability
verdict. A page behind a login is checked with a Playwright storage-state
file and --acknowledge-private-content. Loopback targets need
--allow-private-network; other private addresses stay blocked. Scans are
observations from one cold page load, not test results.

Coding agents get the same checks from 9l mcp as scan_url and
inspect_page, plus generate_playwright_repro, which writes a spec for a
finding that run_test executes and confirm_finding checks across a fix.
These checks were previously the separate qmax-mcp npm server.

What you can do

See the CLI reference, assessment guide, prove, confirm,
native Tier 1 and native Tier 2.
Verified replay and native mobile qualification remain subsequent milestones.
Windows process ownership uses kill-on-close Job Objects; native Windows
amd64 and arm64 CI checks Chromium execution and process-tree cleanup.
Provider CLI shell integrations on Windows remain unqualified.

Add a bounded goal

await n9l.goal('Fill Name using name, then click Continue.', {
  params: {name: 'Fixture Person'}, maxActions: 3, timeoutMs: 30_000,
});
await expect(page.getByRole('heading', {name: 'Review'})).toBeVisible();

Choose --goal-provider anthropic or openai and configure its credential
securely in Go's calling environment, or claude, codex or opencode to
use that CLI's own subscription login without an API key. Offline --goal-script fixtures exercise
the browser/engine contract without paid API calls. Purchase, deletion and
outreach controls stop by default. See goal contracts.

Requirement contracts

Give 9l assess a shared requirements.json with reviewed intent:

{
  "version": 1,
  "requirements": [{
    "id": "checkout-order",
    "reference": "requirements/checkout",
    "revision": "1",
    "expectedOutcomes": [{"id": "order-count", "description": "Checkout displays exactly one order."}]
  }]
}

Put a requirement comment immediately before the test statement and an outcome
comment immediately before the assertion statement:

import {test, expect} from '@playwright/test';

// @9l-requirement checkout-order
test('checkout displays an order', async ({page}) => {
  await page.goto('/checkout'); // uses your project's baseURL
  await page.getByRole('button', {name: 'Place order'}).click();
  // @9l-outcome order-count
  await expect(page.getByTestId('order')).toHaveCount(1);
});

Run 9l assess tests/checkout.spec.ts --requirements requirements.json.
IDs contain 1–80 ASCII letters, digits, _, . or -, starting with a
letter or digit. Requirement IDs are unique; outcome IDs are unique across
the entire contract. This maps source to intent; assessment does not execute
the spec or prove its assertions protect the requirement. See the
assessment guide for mapping rules and limits.

Compatibility

The experimental Android target (Appium with UiAutomator2) ships in 0.6.0.
Install the matching 0.6.0 CLI and SDK together; Android remains opt-in with
the setup and qualification limits below.

Use the matching CLI and SDK release above for the qualified pairing. The
runtime validates 9l.engine/1 and the exact capabilities steps,
artifact-metadata, terminal-outcomes, declared-outcomes (in that order), then validates the
execution evidence. It does not compare package versions or enforce a
same-minor rule. The shared target contract is 9l.target/3; older target
contracts are rejected at the handshake. Mixed package versions are unqualified even when
their contracts match; update both distributions together before diagnosing
a bridge failure. See the SDK contract.

An experimental shared target contract (9l.target/3)
dispatches one deterministic scenario to web and to native targets with the
same app fixture. The experimental Android target
drives an owned local emulator, or an installed app on an explicitly selected
phone, through Appium and UiAutomator2. It supports:

• Native operations: tap, fill, touch at a point, Back, scroll, bounds,
  viewport, keyboard, terminate and activate, with visible, hidden, text,
  state and invalid-state assertions. Controls are reached by resource ID
  (prefixed or bare, as Flutter reports them) or by text, description, hint
  and label aliases.
• Queries that keep the session: isVisible() for a screen that may or
  may not appear, and foreground() to tell a system window from the app;
  both are recorded as actions, never as assertions.
• Fingerprint sign-in (opt-in): on an owned emulator booted from a
  prepared snapshot, fingerprint() touches the virtual sensor; the driver
  never sees a PIN, and phones have no such operation.
• Setup classification: a locked device, a screen that is off, state a
  previous test left behind, or a posture, width or build that doesn't match
  is reported as a setup problem, not an application failure.
• Identity and evidence: the installed build identity, the active display
  and fold posture, and opt-in metadata-only gesture diagnostics.

9l doctor --android checks Java, the SDK, AVDs, devices and the pinned
Appium tooling. Every emulator case runs in CI on hosted Linux, and the
attached-device cases passed on one Android 17 foldable. Target assertions are
reported by the worker and are not independently verified.

Keep your project's existing Playwright version if it is in the qualified
range; upgrading is not required. To check a release locally, see the
development guide.

Add it to your coding agent

The optional 9lives QA skill guides Codex and Claude
through requirements, source provenance, assessment and executed results.
Use the agent setup guide to add it to a project.
For QualityMax-managed tests, the optional
connected QA skill combines local
inspection with hosted crawling, Jev-assisted decisions and cloud verification.

Adjacent QualityMax tools

These are separate distributions. Installing this runner or SDK does not
install or configure them. The Python package also provides a 9l command;
use an explicit executable path when both CLIs are installed.
The installation checks show how to
find stale binaries; MCP setup
checks the version and tools exposed by the host's running server.

Safety and honest limits

Test code executes with the local user's permissions. Use approved targets
and explicitly pass required application environment keys with --pass-env.
Goal parameter values stay in the browser worker; provider credentials stay
in Go. Receipts omit page content and attachment bodies by default, and bound
and sanitize diagnostics. Review the SDK evidence contract
for identity, privacy, skip pins and failure handling.

Architecture

flowchart LR
  Specs[Playwright specs and assertions] --> Go[Go CLI: plan, budgets, scheduling]
  Go --> Worker[TypeScript SDK: fixtures and browser]
  Worker --> Browser[Local Playwright / Chromium]
  Browser --> Worker
  Worker --> Go
  Go --> Receipts[Validated terminal receipts]
  Go -. optional goal decisions .-> Provider[Configured model provider]

Go owns execution identity, cancellation and completeness. TypeScript owns
browser handles and fixtures. The bounded 9l.engine/1 protocol joins them.
See architecture decisions and the
performance baseline, which separates runner overhead
from Playwright and browser costs.

Support and responsible disclosure

Use Discussions for
questions and GitHub Issues
for bugs and feature proposals. See the support guide,
code of conduct, and accessibility statement.
Report vulnerabilities privately using the security policy.

Development

npm ci
npm exec playwright install chromium
npm run demo
go test -race ./...
go vet ./...

The development guide lists SDK qualification, pinned
Python compatibility checks and repository boundaries.
See contributing before proposing changes.

Package and release metadata

The published @9l/playwright 0.6.0
and the Go CLI are separate distributions with one shared version: a single
v<version> tag releases both. Binary archives include LICENSE, NOTICE and
release-wide SHA-256 checksums. See the release runbook
and changelog.

License

Apache-2.0. Third-party components retain their own licenses and
notices. The SDK package and CLI archives include LICENSE and NOTICE.