model-relay

Keep working when Claude hits a 5-hour usage limit, offload subagents,
and switch any window from an interactive panel—without rebuilding 800k prompt caches.

English · 繁體中文 · Install · Features · Configuration · Changelog

Why

Long Claude Code sessions run into four costly routing problems. model-relay solves them on localhost (127.0.0.1:8950) with zero external proxies.

Install

Requires Node.js 20+ and Claude Code 2.1.287 or later (tested on Node.js 22.23.2 and Claude Code 2.1.302).

claude plugin marketplace add cablate/model-relay
claude plugin install model-relay@model-relay

Initialize ~/.claude/model-relay.json, wire ANTHROPIC_BASE_URL into ~/.claude/settings.json, and verify your upstream providers with one command:

cd ~/.claude/plugins/cache/model-relay/model-relay/<version>   # the plugin's install directory (see below)
node tools/setup.mjs init --fix
export ZAI_API_KEY="your-api-key"
node tools/setup.mjs doctor --ping

Every node tools/setup.mjs ... command in this README runs from the plugin's install directory, which is ~/.claude/plugins/cache/model-relay/model-relay/<version>/ ($CLAUDE_CONFIG_DIR/plugins/cache/model-relay/model-relay/<version>/ when you set CLAUDE_CONFIG_DIR; ls that folder for your version). Elsewhere, use the full path to tools/setup.mjs inside it. The command acts on the config directory in $CLAUDE_CONFIG_DIR, or ~/.claude if unset.

env:VAR tokens are read from the router process's own environment, which it inherits from whichever Claude Code session (or setup command) started it. A variable exported after the router started is not seen until the router restarts: stop the router process (doctor prints its pid) and start a new Claude Code session. doctor reads your shell, so it can report the key as set while the router cannot see it.

The example config sends haiku to GLM (matchModels: ["^glm", "haiku"]). As soon as the key is set, every Claude Code haiku call (subagents, WebFetch summaries, internal helpers) goes to z.ai, not only the models you pick. To turn that off, run node tools/setup.mjs alias haiku claude.

Expected output:

Config OK (~/.claude/model-relay.json)
  Port: 8950
  Providers: glm -> glm-5.3[1m], gemini -> gemini-4-argon
  Failover: glm (back to Claude after 10 min)    Retry: on
  Claude settings: ✓ settings.json ANTHROPIC_BASE_URL=http://127.0.0.1:8950
  Router health: {"ok":true,"pid":38780,"failoverSince":0,"nextClaudeTry":0,"failoverProvider":"glm"}
  Upstream ping:
    ✓ glm        (glm-5.3[1m]) -> HTTP 200 in 1950ms
    ✓ gemini     (gemini-4-argon) -> HTTP 200 in 1420ms

Update with claude plugin update model-relay@model-relay.

Who this is for

model-relay is built for Claude Code users who combine a Claude subscription with backup or specialized endpoints (GLM z.ai, DeepSeek, Kimi, MiniMax, OpenRouter, or a local Gemini gateway). If you need a multi-tenant cloud proxy for an entire organization, use LiteLLM instead—model-relay is a single-user localhost daemon.

What's inside

Interactive AbovePrompt control panel and live latency ping

Type /provider (or /provider panel) to open an interactive control panel right above your prompt input box. Click buttons or press ctrl+x tab to use keyboard hotkeys (1–9 to switch routes, p to ping live latency across all providers, r to reset 429 cooldown, f / t / x to flip 429 failover, retry and Remote Control). Switching a pinned session back to Claude requires a two-click confirmation so you never accidentally rebuild a massive prompt cache. Three switch rows turn 429 failover, retry and Remote Control on or off without editing JSON: failover and retry apply from the next request, while Remote Control asks first because it rewrites settings.json, and it only reaches new sessions.

Cache-safe sticky 429 failover

When Anthropic returns a 429 usage-limit response, the router immediately forwards the request to the first provider in failover.chain and pins that session ID for 7 days. After failover.cooldownMs (default 10 minutes), new sessions probe Claude again while existing sticky sessions stay on their backup provider. failover.enabled switches the whole thing off.

Transparent provider retries

A provider answering 429 or 5xx is retried on the SAME provider with exponential backoff (retry.attempts/delayMs/backoff), capped by retry.maxDelayMs; a longer Retry-After keeps polling at that cap. Set retry.attempts: 0 to retry forever (keep hammering until it succeeds — the client connection is watched, so retries stop once the caller disconnects). Claude Code never sees the intermediate failures. retry.enabled turns retrying off everywhere in one go. Turn on retry.failover and a request that still fails once its retries run out is handed to the next provider in failover.chain (or to that provider's own fallback list), never back to one it already tried.

Built-in provider presets and local gateway auto-launch

Use built-in presets (zai, glm, deepseek, kimi, minimax, openrouter, gemini, magpie) so you only need to supply token (or "env:VAR_NAME"). For local gateways on 127.0.0.1, set launch: { command, args } and the router will automatically spawn the gateway process whenever its port is closed.

Model alias routing (pointing haiku / sonnet / opus at a provider)

model-relay separates three routing layers so they never get mixed up:

1. Per-request model alias (matchModels + ANTHROPIC_DEFAULT_*_MODEL): Routes only requests whose model matches (such as haiku subagents or WebFetch summaries) without pinning the main conversation.
2. Per-conversation pin (/provider <name>): Pins only the current conversation ID (x-claude-code-session-id) for 7 days; other windows stay on Claude.
3. 429 failover chain (failover.chain): Takes over only when Anthropic returns a 429 usage-limit error.

Pointing a Claude tier like haiku at a provider (for example glm) coordinates two places at once:

• ~/.claude/settings.json → env.ANTHROPIC_DEFAULT_HAIKU_MODEL = "glm-5.3[1m]" so Claude Code itself resolves the haiku alias to glm-5.3[1m] (including the [1m] context window) and sends "model": "glm-5.3".
• ~/.claude/model-relay.json → providers.glm.matchModels: ["^glm", "haiku"] so the router catches both "glm-5.3" (via ^glm) and any internal Claude Code calls that hardcode "claude-haiku-*" (via "haiku"), rewriting the wire model field to "glm-5.3". This is why the example config sends every Claude Code haiku call to GLM once its key is set; node tools/setup.mjs alias haiku claude undoes it.

Use setup.mjs alias (or /provider alias) to inspect or set both files in one step:

node tools/setup.mjs alias                       # inspect haiku / sonnet / opus / subagent default
node tools/setup.mjs alias haiku glm             # point haiku at glm in both settings.json and model-relay.json
node tools/setup.mjs alias haiku claude          # reset haiku back to Anthropic Claude
node tools/setup.mjs alias subagent sonnet       # set CLAUDE_CODE_SUBAGENT_MODEL=sonnet in settings.json

For per-call subagent routing, model-relay also dynamically registers a full-capability Agent type model-relay:<name> (such as model-relay:glm and model-relay:gemini) via $.agent.register with all parent tools and model set to that provider's model ID, and locks that model in on('agent.spawn')—so you can spawn subagent_type: "model-relay:gemini" on the fly without creating .claude/agents/*.md files or restarting the session.

Edit the config from the command line

set, providers, remove and migrate change the file without hand-editing JSON. Every edit keeps a .relay-backup copy next to the file, tokens are never printed, and the running router picks the change up on its next request. doctor also warns about an old layout, a failover name that matches no provider, and a model name another provider claims first.

node tools/setup.mjs providers
node tools/setup.mjs set failover.chain gemini,glm
node tools/setup.mjs set providers.gemini.fallback none
node tools/setup.mjs migrate --go

Remote Control without giving up the relay

Remote Control only starts when Claude Code talks to api.anthropic.com, which a base URL pointing at the relay breaks. HTTPS proxy mode keeps that address and makes the relay the proxy instead: Claude Code connects through HTTPS_PROXY, the relay answers for api.anthropic.com with a certificate that Claude Code trusts through NODE_EXTRA_CA_CERTS, and model calls are routed exactly as before (providers, sticky failover, retries). Everything else, token refresh and Remote Control traffic included, passes to Anthropic unchanged.

node tools/setup.mjs proxy enable --go
node tools/setup.mjs rc

• Contained trust: The CA the relay creates can only vouch for api.anthropic.com (a critical name constraint), its private key is discarded right after signing, and only Claude Code processes that carry NODE_EXTRA_CA_CERTS load it. The system certificate store is never touched.
• Safe fallback: Without a usable certificate the relay tunnels api.anthropic.com untouched, so Claude Code keeps working, just unrouted. A stopped router is different: the plugin's automatic direct fallback only works in ANTHROPIC_BASE_URL mode, so in proxy mode Claude Code cannot connect until the router is restarted (node tools/setup.mjs rc).
• Reversible: proxy disable --go restores ANTHROPIC_BASE_URL, and the previous settings.json is kept as settings.json.relay-backup.

HTTPS_PROXY lives in settings.json env, so it also applies to commands Claude Code runs (git, curl, gh): they go through the relay as blind tunnels, and fail while the router is down.

Configuration

Configuration lives in ~/.claude/model-relay.json (or legacy ~/.claude/custom-agent.json). The directory defaults to $CLAUDE_CONFIG_DIR when set, falling back to $USERPROFILE/.claude or $HOME/.claude.

Commands

Cost, privacy, and permissions

• Network and data flow: The router listens only on 127.0.0.1:<port>. Requests go strictly to claudeUrl (https://api.anthropic.com) or the baseUrl endpoints you configure. /provider ping sends a max_tokens: 1 request to each configured provider when you trigger it. No telemetry or analytics exist.
• Local files: Reads ~/.claude/model-relay.json (or custom-agent.json) and writes session sticky state to ~/.claude/model-relay-state.json plus local logs (custom-agent-router.log and custom-agent-requests.log, each emptied once it passes 5 MB).
• Local certificate (proxy mode only): ~/.claude/model-relay-pki/ holds a public CA file and one leaf certificate with its key (restricted to your user). The CA is limited to api.anthropic.com and trusted only through NODE_EXTRA_CA_CERTS; delete the folder or run proxy disable --go to stop using it.
• Plugin permissions: Inspect all hooks, state keys, and env access at any time by running claude plugin validate /path/to/model-relay.

Supported environments

Limitations

• Requires an Anthropic-compatible /v1/messages endpoint: Providers that only speak OpenAI Chat Completions require a local translation proxy (and can be supervised via providers.<name>.launch).
• Switching back to Claude mid-session rebuilds cache: Because prompt caches do not transfer across providers, /provider claude on a long conversation will resend the full conversation history to Anthropic.

Troubleshooting

Uninstall

1. Run claude plugin uninstall model-relay@model-relay and remove ANTHROPIC_BASE_URL from ~/.claude/settings.json (in proxy mode remove HTTPS_PROXY, NODE_EXTRA_CA_CERTS and NO_PROXY instead; running node tools/setup.mjs proxy disable --go first does that and puts ANTHROPIC_BASE_URL back, which you then remove).
2. Delete optional state files (~/.claude/model-relay.json, ~/.claude/model-relay-state.json, ~/.claude/model-relay-pki/, ~/.claude/custom-agent-router.log, ~/.claude/custom-agent-requests.log).

Run from source

node tools/check.mjs
claude plugin validate .
claude plugin test .

See CONTRIBUTING.md for architecture and release steps.

More

• Changelog
• Contributing
• Security Policy
• License (MIT)