decisions-judge-mcp

Typed decisions for AI agents, as an MCP tool. Ask yes/no probability (noul), choice among options, or score on ordered levels about any JSON application state — all questions answered in one fast request.

Backed by the TypeSafe System One model (Jev) via @typesafe-ai/sdk, with an opt-in Cloudflare Workers AI provider — see Providers.

Get started

Add to claude_desktop_config.json (~/Library/Application Support/Claude/) — the same JSON shape works for codex, goose, and pi:

{
  "mcpServers": {
    "decisions-judge": {
      "command": "npx",
      "args": ["-y", "decisions-judge-mcp"],
      "env": { "TYPESAFE_API_KEY": "your-key-here" }
    }
  }
}

Then ask your agent anything answerable with a judgment — it can call the judge tool.

Remote transport (opt-in)

By default the server speaks stdio. Set JUDGE_TRANSPORT=http to serve the MCP Streamable HTTP transport (spec revision 2026-07-28) at POST /mcp instead:

JUDGE_TRANSPORT=http HTTP_HOST=127.0.0.1 HTTP_PORT=8080 npx -y decisions-judge-mcp

• JUDGE_TRANSPORT: stdio (default) or http. Any other value exits with an error.
• HTTP_HOST: bind address, default 127.0.0.1.
• HTTP_PORT: port, default 8080; must be 1-65535.

Serving is stateless: each request is handled independently, with no protocol-level sessions (GET/DELETE on /mcp return 405).

A GET /health endpoint returns 200 with {provider, transport, keys} for liveness checks; it does not shadow /mcp routing.

Security warning: the HTTP endpoint is unauthenticated. Production remote deployments must be fronted by an authenticating OAuth 2.1 proxy per the MCP 2026-07-28 authorization specification.

Architecture

Demo

Judge returns typed answers your code can branch on — and a guaranteed fallback envelope when the model is unreachable.

See CONTRIBUTING.md to regenerate the demo.

Example

// judge tool call
{
  "state": { "tests": "passing", "lint": "clean", "filesChanged": 3 },
  "questions": {
    "ready_to_merge": {
      "type": "noul",
      "instructions": "Is this change safe to merge?"
    },
    "next_step": {
      "type": "choice",
      "instructions": "What should the agent do next?",
      "criteria": {
        "merge": "create the merge commit",
        "iterate": "keep refining the change",
        "escalate": "hand back to the human"
      }
    }
  }
}

// response
{
  "answers": {
    "ready_to_merge": { "type": "noul", "noul": 0.93 },
    "next_step": {
      "type": "choice",
      "choice": "merge",
      "confidence": 0.97,
      "probabilities": { "merge": 0.97, "iterate": 0.02, "escalate": 0.01 }
    }
  },
  "model": "jev-latest",
  "usage": { "input_tokens": 214, "output_tokens": 18 },
  "fallback": false
}

Quickstart

Requires Node >= 20 and at least one TypeSafe API key in the environment via TYPESAFE_API_KEYS, TYPESAFE_API_KEY, or TYPESAFE_API_KEY_2 .. TYPESAFE_API_KEY_9.

npx -y decisions-judge-mcp          # latest
npx -y [email protected]    # pinned, for supply-chain reproducibility

Configuration

Invalid JUDGE_PROVIDER values, or cloudflare-workers-ai selected with missing
Cloudflare credentials, fail fast at startup (non-zero exit naming the offending
variable).

Selecting typesafe-api with no key configured prints a startup warning (the
server still runs; judge calls return fallback envelopes) instead of exiting.
When multiple keys are configured, a rate-limited request (HTTP 429/529) is
rotated to the next key before any fallback envelope is returned.

MCP client configuration

All examples use npx -y, the standard pattern for Node-based MCP servers. Swap in decisions-judge-mcp@<version> in the args if you prefer pinning.

Claude Code (one command):

claude mcp add --scope user decisions-judge \
  --env TYPESAFE_API_KEY=... -- npx -y decisions-judge-mcp

goose (~/.config/goose/config.yaml):

mcp:
  servers:
    decisions-judge:
      command: npx
      args:
        - -y
        - decisions-judge-mcp
      env:
        TYPESAFE_API_KEY: "..."

pi (~/.config/pi/mcp.json or equivalent):

{
  "mcpServers": {
    "decisions-judge": {
      "command": "npx",
      "args": ["-y", "decisions-judge-mcp"],
      "env": { "TYPESAFE_API_KEY": "..." }
    }
  }
}

Tool reference: judge

Instructions and criteria values accept either plain strings or arbitrary JSON
structure (objects/arrays). In-flight requests are cancelled when the client
disconnects.

Output: {answers, model, usage} on success, {fallback: true, error} on failure — failures never block agent workflows. A noul answer is a probability, not a verdict: pick your own action threshold in the caller (e.g. proceed only above 0.7).

Writing good questions

Jev answers short, well-framed questions better than terse ones. A working reference implementation (LamplighterPaul/jev-piano) demonstrates the idiom.

Four patterns, all applicable to the three question types (noul, choice, score):

1. One decision per question ID. Each entry in questions asks exactly one concept. Don't bundle "is it correct and should we ship it" into a single noul — ask two questions.
2. Descriptive criteria strings. Every choice option is described by its own intrinsic properties and trade-offs, never by position, index, or an endorsement of the desired answer. Describing one option as "lowest risk" and its rivals as problems is a hint; describing each option's genuine trade-off is not. The model should reach the answer from the state plus balanced descriptions.
3. Inline state framing. Name the relevant state inside the question text ("It is a G7 chord, moving next to Cm...") rather than relying on the caller to correlate a separate state blob. The state field is still sent — inline framing just makes each question self-contained.
4. Distinct framing per role, one step ahead. The same kind of decision at different points gets differently phrased questions (e.g. opening vs. closing), because the decision's job differs. When sequencing matters, ask what the next decision should prepare for — without that, decisions collapse onto the default option.

Concrete example, rewritten from the terse version in Example to follow the idiom:

// judge tool call
{
  "state": { "tests": "passing", "lint": "clean", "filesChanged": 3, "reviewComments": 0, "branchAgeDays": 2, "behindMainBy": 7 },
  "questions": {
    "ready_to_merge": {
      "type": "noul",
      "instructions": "CI is green (tests passing, lint clean), 3 files changed, and no review comments are open. Is this change safe to merge to main right now?"
    },
    "next_step": {
      "type": "choice",
      "instructions": "The branch is 2 days old and has drifted 7 commits behind main. What should the agent do next?",
      "criteria": {
        "merge": "rebase onto main and create the merge commit now, while CI is green",
        "iterate": "continue refining on the branch before any integration step",
        "escalate": "hand the decision back to the human with a summary of the state"
      }
    }
  }
}

Note what changed: the state is restated inline and consistently, the criteria are balanced, and the framing names the role of this decision — each rule applied once, concretely.

Prompts

The server exposes three static decision-pattern prompt templates: verify-claim (atomic yes/no checks of a claim against state), classify (pick one option with balanced criteria), and route (decide what the next decision should prepare for). Each template returns a ready-to-send judge call skeleton with atomic questions and structured {summary, signals} criteria. Every template is version-stamped (wording-v1); rewording bumps the stamp so earlier calibration runs stay comparable. Fetch one with prompts/get:

{ "jsonrpc": "2.0", "id": 2, "method": "prompts/get", "params": { "name": "verify-claim", "arguments": { "state": "{}" } } }

The response is a single user message whose text embeds a judge skeleton like:

{
  "tool": "judge",
  "arguments": {
    "state": "<state JSON goes here>",
    "questions": {
      "claim_supported": {
        "type": "noul",
        "instructions": "The provided state contains evidence that directly supports the claim under review. Does the evidence support the claim?"
      }
    }
  }
}

Fill in the state and send it to the judge tool.

Providers

Selected once at startup via JUDGE_PROVIDER (default typesafe-api). The judge tool's schema and behavior are identical under both providers.

• typesafe-api (default): uses @typesafe-ai/sdk with one or more TypeSafe API keys (TYPESAFE_API_KEYS CSV, TYPESAFE_API_KEY, or TYPESAFE_API_KEY_2 .. TYPESAFE_API_KEY_9); keys are tried in order with rotation on rate limits. See Configuration for details.
• cloudflare-workers-ai: routes the same Jev model through Cloudflare Workers AI; requires the two Cloudflare variables above.

Billing gotcha: typesafe/jev is a partner model on Cloudflare, so runs are metered from the AI Gateway prepaid credit balance, not the account's payment card. Accounts with a valid card but zero credit balance get HTTP 402 (error 2021, "Insufficient balance"); top up AI Gateway prepaid credit before calling.

Development

npm ci
node --check server.mjs
node scripts/smoke.mjs         # MCP stdio initialize handshake
node scripts/smoke-judge.mjs   # judge fallback envelope (no API key needed)
node server.mjs                # stdio server; needs TYPESAFE_API_KEY to answer

Flagship consumer: clouatre-labs/agentic-coder-skill.

License

Apache-2.0, see LICENSE.