SVDO Meter

SVDO Meter is an open-source telemetry and evaluation layer for AI coding agents. Measure execution, token usage, cost, sessions, tooling behavior, and repository alignment across Codex, Claude Code, and other coding agents.

It associates a ticket/work identifier with an agent CLI run, invokes or resumes the selected harness, normalizes objective events where possible, and writes durable append-only telemetry locally. It is not an orchestration framework, ticketing system, model router, or remote connector.

Keywords: ai-agents, coding-agents, observability, llm, codex, claude-code, developer-tools.

Current Status

The v0.1 baseline centers on:

svdo-meter run
svdo-meter eval run
svdo-meter compare
svdo-meter telemetry
the codex, claude, and opencode harnesses
local per-run JSONL telemetry under .svdo/meter/
repository alignment eval definitions under .svdo/evals/
comparison reports that combine telemetry with JSON artifacts under .svdo/runs/ and .svdo/evals/
optional live stdout NDJSON event streaming from svdo-meter run
rebuildable session association from session.discovered events
fixture-based tests that do not require live Codex execution

Help

The CLI is built with Clap, so command help is available from the binary:

svdo-meter --help
svdo-meter run --help
svdo-meter eval --help
svdo-meter eval run --help
svdo-meter compare --help
svdo-meter report --help
svdo-meter telemetry --help

If running from source:

cargo run -p svdo-meter -- --help
cargo run -p svdo-meter -- run --help
cargo run -p svdo-meter -- eval --help
cargo run -p svdo-meter -- eval run --help
cargo run -p svdo-meter -- compare --help
cargo run -p svdo-meter -- report --help
cargo run -p svdo-meter -- telemetry --help

Install From GitHub

Install the latest published release binary without cloning the repository:

curl -fsSL https://raw.githubusercontent.com/brianofrokk3r/svdo-meter/main/install.sh | bash

The installer downloads the matching GitHub Release asset, verifies its SHA-256 checksum, installs svdo-meter to:

$HOME/.local/bin

Override the install directory with SVDO_METER_INSTALL_DIR:

curl -fsSL https://raw.githubusercontent.com/brianofrokk3r/svdo-meter/main/install.sh | SVDO_METER_INSTALL_DIR="$HOME/bin" bash

Supported installer platforms:

Linux x86_64
macOS x86_64
macOS arm64/aarch64

After installation, the script verifies the binary with svdo-meter --help. If $HOME/.local/bin is not on PATH, add it before running svdo-meter from another directory.

30-Second Example

Run one measured agent session from a repository workspace, then render the local report:

svdo-meter run \
  --ticket ENG-142 \
  --label "Add password reset flow" \
  --harness codex \
  --workspace ~/code/app \
  "Implement the password reset flow described in ENG-142"

svdo-meter report ENG-142 --workspace ~/code/app
svdo-meter compare ENG-142 --workspace ~/code/app

For measured runs, --ticket, --ticket-id, and --id are equivalent spellings for the same ticket/work identifier. The value is recorded as telemetry ticket_id and used for session association, reports, and comparisons.

SVDO Meter writes append-only telemetry to:

~/code/app/.svdo/meter/<run-id>.jsonl

To keep meter artifacts outside the source workspace, pass an explicit output directory to svdo-meter run:

svdo-meter run \
  --ticket ENG-142 \
  --harness codex \
  --workspace "$SVDO_WORKSPACE" \
  --output-dir "$SVDO_TMPDIR/meter" \
  "Implement ENG-142"

When --output-dir is omitted, the default remains <workspace>/.svdo/meter/.

The report output looks like this fixture-backed example:

SVDO Trace
────────────────────────────

Work
  ENG-142

Harness
  codex

Session
  019c8a-fixture

Runs
  2

Agent Time
  18m 42s

Tokens
  Input   120,000
  Output  42,000
  Cache   32,213
  Total   194,213

svdo-meter compare finds runs for the same ticket/work id and compares them by harness and/or model:

SVDO Comparison — ENG-142
────────────────────────────────────────────────────────

         Claude  Codex   OpenCode
Harness  Claude  Codex   OpenCode
Model    sonnet  gpt-5   gpt-5
Runs     3       3       3

Performance
  Success rate  100%    100%    67%
  Agent time    10m 42s 12m 18s 16m 05s
  Turns         14      18      25
  Commands      14      17      —
  Tool calls    27      31      42

Compile

Prerequisites:

Rust stable toolchain, preferably installed with rustup
Cargo on PATH
codex on PATH when running the Codex harness for real work
claude on PATH and authenticated when running the Claude Code harness for real work
opencode on PATH when running the OpenCode harness for real work

Harness API Keys and CI Secrets

For measured harness runs, SVDO Meter shells out to the selected harness CLI and lets that child process read credentials from its environment. Configure and validate codex, claude, or opencode the same way you would before running that CLI directly. TypeSafe-backed eval judging is the exception: svdo-meter eval run --judge-backend typesafe calls the TypeSafe System One API directly and reads its API key from TYPESAFE_API_KEY by default.

Harness	Typical credential environment
Codex CLI	`OPENAI_API_KEY`; optional `OPENAI_PROJECT_ID`
Claude Code CLI	`ANTHROPIC_API_KEY`
OpenCode CLI	Provider-specific variables, commonly `OPENAI_API_KEY`, optional `OPENAI_PROJECT_ID`, or `ANTHROPIC_API_KEY` depending on the configured model provider

Never commit API keys, tokens, .env files, shell history, or generated credential files to the repository. For CI setup, secure-variable guidance, Bitbucket Pipelines, and GitHub Actions references, see CI Secrets and Harness Credentials.

Build the debug binary:

cargo build -p svdo-meter

Run it from the build output:

./target/debug/svdo-meter --help
./target/debug/svdo-meter run --help
./target/debug/svdo-meter eval run --help
./target/debug/svdo-meter compare --help
./target/debug/svdo-meter report --help
./target/debug/svdo-meter telemetry --help

Build the optimized release binary:

cargo build --release -p svdo-meter

Run the release binary:

./target/release/svdo-meter --help

Install it into Cargo's bin directory:

cargo install --path crates/svdo-meter

After install, make sure Cargo's bin directory is on PATH, then run:

svdo-meter --help

Run Codex With Telemetry

svdo-meter run \
  --ticket ENG-142 \
  --label "Add password reset flow" \
  --harness codex \
  --workspace ~/code/app \
  "Implement the password reset flow described in ENG-142"

The ticket/work id may also be supplied as --ticket-id ENG-142 or --id ENG-142; all forms map to the same telemetry and reporting field.

Conceptual Codex invocation:

codex exec --json -C ~/code/app "Implement the password reset flow described in ENG-142"

Use --dangerous-bypass only when the selected harness should bypass approval and sandbox protections. SVDO Meter records that posture on run.started telemetry.

Use --codex-skip-git-repo-check with the Codex harness when intentionally measuring work in a disposable or otherwise untrusted workspace that is not a git repository.

Run Claude Code With Telemetry

Claude Code runs use non-interactive print mode with stream JSON so SVDO Meter can normalize live events without a TTY:

svdo-meter run \
  --ticket ENG-142 \
  --label "Add password reset flow" \
  --harness claude \
  --model sonnet \
  --workspace ~/code/app \
  "Implement the password reset flow described in ENG-142"

Conceptual Claude Code invocation:

claude -p "Implement the password reset flow described in ENG-142" --output-format stream-json --verbose --model sonnet

Supported Claude-specific options include --claude-permission-mode, --claude-allowed-tool, --claude-disallowed-tool, --claude-add-dir, --claude-mcp-config, --claude-strict-mcp-config, --claude-settings, --claude-setting-sources, system prompt flags, --claude-max-turns, and --claude-max-budget-usd.

Run OpenCode With Telemetry

OpenCode runs use non-interactive opencode run mode with JSON output requested for machine-readable telemetry.

svdo-meter run \
  --ticket ENG-142 \
  --label "Add password reset flow" \
  --harness opencode \
  --model github-copilot/gpt-5 \
  --opencode-agent build \
  --workspace ~/code/app \
  "Implement the password reset flow described in ENG-142"

Conceptual OpenCode invocation:

opencode run --format json --dir ~/code/app --model github-copilot/gpt-5 --agent build "Implement the password reset flow described in ENG-142"

Use --opencode-agent <AGENT> to pass an OpenCode agent name through as opencode run --agent <AGENT>.
Use --dangerous-bypass only when OpenCode should run with automatic execution; SVDO Meter maps that posture to OpenCode --auto and records it on run.started telemetry.
When --session <SESSION> is supplied, SVDO Meter maps it to opencode run --session <SESSION>. If durable telemetry already contains a discovered OpenCode session for the same ticket, harness, and workspace, later runs can reuse that session through the same OpenCode session flag.

For longer or reusable instructions, read the prompt from a UTF-8 text file:

svdo-meter run \
  --ticket ENG-142 \
  --label "Add password reset flow" \
  --harness codex \
  --workspace ~/code/app \
  --prompt-file prompts/eng-142.md

To pipe live normalized events to another process, enable stdout NDJSON output. Durable JSONL telemetry is still written, using --output-dir when provided:

svdo-meter run \
  --ticket ENG-142 \
  --harness codex \
  --workspace ~/code/app \
  --emit ndjson \
  "Implement ENG-142" | my-company-ingester

Equivalent sink selection is available with --sink stdout; --sink jsonl explicitly selects the durable local JSONL sink. Supplying both --sink stdout and --emit ndjson produces one stdout event stream, not duplicate records.

Configure Run Output Directory

svdo-meter run writes durable JSONL telemetry under <workspace>/.svdo/meter/ by default. Use --output-dir <PATH> when a container or worker should keep meter artifacts in an infrastructure-owned directory instead:

svdo-meter run \
  --ticket ENG-142 \
  --harness codex \
  --workspace "$SVDO_WORKSPACE" \
  --output-dir "$SVDO_TMPDIR/meter" \
  --prompt-file "$SVDO_METER_CONFIG/prompt.md"

This works well with worker launchers that append meter arguments through SVDO_METER_EXTRA_ARGS, for example:

export SVDO_METER_EXTRA_ARGS="--output-dir $SVDO_TMPDIR/meter"

The directory is created before telemetry is written. If the path cannot be created or is not usable as a directory, the run fails with an error that includes the target path.

--output-dir only changes where svdo-meter run writes telemetry. Existing svdo-meter report, compare, eval, and telemetry commands continue to read their documented workspace locations such as <workspace>/.svdo/meter/, <workspace>/.svdo/evals/, and <workspace>/.svdo/runs/.

Resume A Known Session

SVDO Meter records a session.discovered event when the selected harness exposes a session/thread ID. Later runs for the same ticket, harness, and workspace automatically use the latest known session when available.

To override automatic lookup:

svdo-meter run \
  --ticket ENG-142 \
  --label "Add password reset flow" \
  --harness codex \
  --session 019c8a42-f72... \
  --workspace ~/code/app \
  "Fix the remaining tests"

For Claude Code, --session maps to claude --resume <session> in print mode. Claude-specific controls are also available:

svdo-meter run --ticket ENG-142 --harness claude --claude-continue "Run the remaining tests"
svdo-meter run --ticket ENG-142 --harness claude --claude-resume auth-refactor "Finish this PR"
svdo-meter run --ticket ENG-142 --harness claude --claude-resume auth-refactor --claude-fork-session "Try an alternate fix"

For OpenCode, --session maps to opencode run --session <session> while preserving JSON output and model pass-through:

svdo-meter run --ticket ENG-142 --harness opencode --session ses_abc123 "Continue this fix"

Select A Model

Models are harness-specific configuration, not separate adapters:

svdo-meter run \
  --ticket ENG-142 \
  --harness codex \
  --model gpt-5 \
  --workspace . \
  "Implement ENG-142"

For Claude Code, --model maps to claude --model and accepts Claude Code aliases or full model identifiers supported by the installed Claude CLI. For OpenCode, --model maps to opencode run --model and accepts provider/model values supported by the installed OpenCode CLI.

Estimate Token Cost

svdo-meter report can estimate local token cost from a UTF-8 JSON pricing file supplied through the CLI. Rates are cost per 1,000,000 tokens and are keyed by exact model identifier:

svdo-meter report ENG-142 --pricing-file pricing.json

When telemetry references a model that is not present in the pricing JSON, the report marks that model's cost as unavailable instead of using a default price.

Compare Runs

svdo-meter compare compares existing telemetry and eval results across runs. Give it a ticket/work id for a focused comparison:

svdo-meter compare ENG-142 --workspace ~/code/app

Compare recent aggregate performance by harness:

svdo-meter compare \
  --harness codex \
  --harness claude \
  --harness opencode \
  --since 30d

Compare models within a harness:

svdo-meter compare \
  --harness opencode \
  --model openai/gpt-5.6 \
  --model anthropic/claude-sonnet-5

Compare harnesses for one model:

svdo-meter compare \
  --model gpt-5.6 \
  --harness codex \
  --harness opencode

The comparison report derives canonical run-summary style records from .svdo/meter/*.jsonl, then enriches matching runs from JSON artifacts under .svdo/runs/ and .svdo/evals/ when those files are present. It reports performance, cost, quality, and efficiency metrics. Missing observability is shown as —; true observed zeroes remain visible as 0.

Examples

Full runnable examples live in docs/examples.md, including a label-grouped reporting walkthrough for plan and implement phases, plus the calculator benchmark that compares Codex gpt-5.5 with OpenCode openai/gpt-5.5 through run, eval run, report, and compare.

Apply SVDO Meter To A Repository

SVDO Meter fits best as a lightweight operating layer for AI-assisted repository work:

Observe agent work by wrapping meaningful Codex, Claude Code, or OpenCode sessions with svdo-meter run.
Align the repository by defining deterministic checks and optional judge checks under .svdo/evals/ and .svdo/standards/.
Govern stable expectations by running trusted evals in CI and publishing reports or artifacts for review.

A typical workflow is:

svdo-meter eval run repo-alignment

svdo-meter run \
  --ticket ENG-142 \
  --label "Add password reset flow" \
  --harness codex \
  --workspace . \
  "Implement the password reset flow described in ENG-142"

svdo-meter eval run repo-alignment
svdo-meter report ENG-142
svdo-meter compare ENG-142

Use pre-commit hooks only for fast deterministic evals such as formatting, linting, schema checks, or quick unit tests. Run broader deterministic evals in CI on pull requests. Treat LLM judge checks as advisory at first, then make them blocking only after the standards and scoring behavior are stable enough for the team.

For the fuller rollout guide, see docs/adoption.md.

Run Repository Alignment Evals

Repositories can define reusable alignment evals and standards under .svdo/:

.svdo/
  meter/
  runs/
  evals/
  standards/

Run all evals in the current repository:

svdo-meter eval run

Run one eval by id, file stem, or file name:

svdo-meter eval run add-account-endpoint
svdo-meter eval run add-account-endpoint.yaml

Run evals against another repository without changing directories:

svdo-meter eval run --workspace ~/code/app

Eval output supports the same report-style formats:

svdo-meter eval run --format terminal
svdo-meter eval run --format json
svdo-meter eval run --format csv

Run judge checks with an LLM judge:

svdo-meter eval run --harness codex --model gpt-5
svdo-meter eval run api-contract --harness codex --model gpt-5
svdo-meter eval run --harness claude --model sonnet
svdo-meter eval run api-contract --harness opencode

Command checks run locally in the selected workspace. Judge checks are skipped unless --harness codex, --harness claude, or --judge-command is configured. The Codex and Claude judge paths invoke the selected CLI with the selected model and ask for a JSON score. Codex judge runs include Codex's git-repository check bypass so disposable eval workspaces do not need to be git repositories; normal svdo-meter run --harness codex invocations are unchanged. Custom judge commands receive the request JSON path as their final argument, also available as SVDO_METER_JUDGE_REQUEST, and must print JSON containing a score from 0.0 to 1.0 plus optional passed, violations, usage, model, harness, and session metadata.

Telemetry checks can assert that expected local .svdo/meter/ events occurred in the latest run:

checks:
  - id: requires-apply-patch
    type: telemetry
    event_type: tool.started
    tool_name: apply_patch
    min_count: 1

Telemetry

By default, events are written to:

.svdo/meter/<run-id>.jsonl

For measured runs, --output-dir <PATH> writes those per-run JSONL files to the configured directory instead. This is intended for containerized worker runs that mount or collect meter artifacts separately from the project source tree.

Each line is one immutable canonical JSON event. The selected meter directory is the source of truth; session registries and reports are rebuildable projections.

The local JSONL sink is enabled by default and remains active when svdo-meter run --sink stdout or svdo-meter run --emit ndjson is used.

Raw provider payloads are not persisted by default. This avoids storing prompts, model responses, command output, tool results, environment variables, secrets, and other sensitive content unless raw retention is explicitly enabled in code/configuration.

Conformance Fixtures

Provider conformance fixtures live under tests/fixtures/<provider>/. Keep the .jsonl files as captured or synthetic provider event streams, and record capture context in the matching .metadata.yaml sidecar when a fixture is used for provider conformance.

Provider fixture metadata is required to include:

provider: codex
cli_version: 0.154.0
schema_observed: 2026-09-11
model: gpt-5

When refreshing fixtures, set provider to the harness name, set cli_version from the provider CLI used for capture, for example codex --version or opencode --version, set schema_observed to the date the output shape was observed, and record the model used for capture. The conformance tests validate required metadata before replaying fixtures and do not launch live provider CLIs.

Inspect local telemetry without modifying .svdo/meter/:

svdo-meter telemetry sessions --workspace ~/code/app
svdo-meter telemetry runs --workspace ~/code/app
svdo-meter telemetry inspect 018f6f1b-97f1-7c04-9a96-111111111111 --workspace ~/code/app
svdo-meter telemetry inspect sess-abc123 --workspace ~/code/app

The telemetry inspection commands tolerate missing or empty telemetry files, report malformed JSONL lines with line numbers, and call out missing token fields on token-bearing records.

svdo-meter