skilled

Repeatable tests for agent skills, with a clear path from a failed case to a verified improvement.

Install · First check · Use with your agent · Documentation

Write down what a Skill should do, run those cases against your agent, and keep the reports. After a change, rerun the same cases and compare results to see what improved and what regressed.

Component	What it does	How you use it
`skilled`	Validates Skills, runs evaluations, and compares saved reports	CLI commands or MCP tools
`skilled-improve`	Guides your agent through diagnosing failures, revising a Skill, and checking the result	Bundled MCP prompt or host workflow

One installation includes both. skilled-improve is a workflow, not a second terminal command. Validation, dry runs, and saved-report comparison work without model credentials. Live evaluations use the engine and account you configure.

Install

macOS & Linux

curl -fsSL https://raw.githubusercontent.com/DeadHead04/skillEd/main/install.sh | bash
export PATH="$HOME/.local/bin:$PATH"
skilled --version

Add the PATH line to ~/.zshrc or ~/.bashrc for future terminals. Requires Bash, curl, and standard archive tools.

Windows PowerShell

$installer = Join-Path $env:TEMP 'skilled-install.ps1'
Invoke-WebRequest https://raw.githubusercontent.com/DeadHead04/skillEd/main/install.ps1 -OutFile $installer
& $installer
$env:Path = "$(Join-Path $HOME '.local/bin');$env:Path"
skilled --version

Add $HOME\.local\bin to your user PATH for future terminals. Follow your organization's script policy if PowerShell blocks downloaded scripts.

The installer downloads the latest release, verifies its SHA-256 checksum, and installs to ~/.local/bin. The Node runtime and application dependencies are bundled. No GitHub login or source build is needed.

Manual downloads · Pin a version, update, or uninstall · Platform verification

Prefer having your agent handle setup? Paste this into your agent:

Set up skilled in this project using https://raw.githubusercontent.com/DeadHead04/skillEd/main/instruction.md, then validate a first evaluation without calling a model.

Your first check

For a project that already has a SKILL.md and evals/eval.yaml, run these commands from its root:

skilled validate --skill .
skilled validate evals/eval.yaml
skilled run evals/eval.yaml --dry-run

This checks the Skill, evaluation configuration, and case selection without calling a model. Starting from scratch? The Hello World walkthrough includes every file to create and works on all three operating systems.

Once your chosen engine is installed and authenticated, run the evaluation:

skilled run evals/eval.yaml --format json --format html --format junit

Open the HTML report in the directory printed by the CLI to inspect case verdicts and evidence. Live evaluations may incur provider charges; see engine and credential setup.

Improve with your agent

From your project folder, activate the bundled workflow for your host:

skilled setup --host codex
# Or:
skilled setup --host claude-code

Start a new host session in that project, then ask: Use skilled-improve to evaluate this Skill.
Setup copies the canonical workflow into the host's project Skill directory. It preserves
host settings and refuses to overwrite local workflow edits. Repeat the command after
upgrading skilled; use --remove to remove an unmodified managed copy. Existing engine
logins work for local evaluation; Docker and a separate API key are optional.

For tool-based integration, use the optional MCP setup guide. The server command is:

skilled mcp --workspace /absolute/path/to/your-project

Then ask your agent:

Use skilled-improve to validate this Skill and its evaluation suite. Explain the problems and suggest regression cases. Start without calling a model.

The default MCP connection supports validation, dry runs, comparison, and the bundled workflow. To execute evaluations through MCP, configure the engine and add --allow-execution to the server arguments.

The improvement loop

Evaluate. Run versioned cases and preserve the report as a baseline.
Inspect. Review failed assertions and the actual outputs.
Revise. Use skilled-improve to help your agent edit the Skill or add missing cases.
Compare. Rerun the same suite and check the saved reports. See the scoring contract for skipped checks and compatible baselines:

skilled compare baseline/iteration-1/report.json candidate/iteration-1/report.json --fail-on-regression

Use the report paths from your runs. Comparison checks observed changes and flags coverage differences; it does not continuously monitor conversations. How comparison and scoring work →

Built for repeatable evaluation

Capability	What you get
Versioned YAML cases	Prompts, fixtures, and expected behavior beside your Skill
Flexible grading	Assertions, custom scripts, and model judges
Your agent engine	Claude Code, Codex, OpenCode, Qoder CLI, Qwen Code, or a custom engine
Execution choices	Host, Docker, or a configured OpenSandbox server
Reports you can use	JSON for tooling, HTML for review, and JUnit for CI
Baselines and repetition	With/without-Skill evaluations and repeated runs
Existing evaluation suites	Import Anthropic `evals.json` or use automatic discovery

Host observation capture is a separate, optional integration. See the Codex plugin or DSH plugin for setup.

Find your next step

I want to…	Guide
Try a complete example	Your first evaluation
Connect my agent	MCP tools and skilled-improve
Write cases and choose a judge	Writing evaluations
Check for regressions	Comparison and scoring
Look up commands or exit codes	CLI reference
Contribute or run the tests	Contributing

Development

Use Node 24.21.0 (see .nvmrc), npm, and Git:

git clone https://github.com/DeadHead04/skillEd.git
cd skillEd
npm ci
npm run build
node dist/src/bootstrap.js --help

The application is TypeScript compiled to ESM. src/ contains the CLI and execution API; skills/ and plugins/ contain agent workflows; examples/, tests/, and docs/ contain examples, verification, and guides. Contribution checks and real-service testing →

Release and verification

skilled is an early-stage project. Release 0.2.0 includes the MCP server, Skill integrity checks, and saved-report comparison. Distribution is through GitHub Releases; an npm registry package and Homebrew formula are not published.

Packaged execution and installation have been tested on Linux, macOS, and Windows. Native architecture coverage, authenticated model scenarios, and service limits are documented in verification and release scope. Archive availability is not a claim that every device or provider combination has been tested.

Report a bug · Changelog · Security policy

License

Apache-2.0. Original copyright and attribution notices are retained in NOTICE.