Tokenade

Cut your AI coding agent's token bill — locally, automatically, on every prompt.

Install · What it does · Benchmark · Website

Tokenade is a single Rust binary that sits between your coding agent (Claude
Code, Codex, Cursor, Copilot CLI, OpenCode, Kilo Code, Qwen Code…) and the
model. It strips what the model never needed to see — 2,000-line build logs,
whole files when one function mattered, verbose MCP payloads, re-reads of
content already in context — before it is billed, and keeps every byte it
folds recoverable on demand.

#1 on THOL

Tokenade ranks #1 on THOL, the Token-Harness Optimizer Leaderboard —
real end-to-end agent runs, every optimizer measured the same way:
39% cheaper on long sessions. Method and full results:
https://tokenade.net/en/thol.

What it does

• Command compaction — test runs, builds, logs, git, kubectl,
  docker, package managers… noisy output is folded to what matters
  (failures, numbers, decisive lines) before the agent reads it.
• Read deduplication — content the agent already has comes back as a
  short reference, or as a diff when it changed.
• Code intelligence — map, query, skeleton, impact and hybrid
  semantic search, so the agent stops reading whole files to find one thing.
• MCP optimization — MCP servers are wrapped: responses compacted in
  transit, tool manifests trimmed to their callable skeleton.
• Web — tokenade web / tokenade search return compact Markdown and
  result lists instead of raw HTML.
• Context and output — prompt-cache-aware compaction of stale context,
  and a lean-output style note that drops filler without dropping facts.
• Secret redaction — credentials are scrubbed from what Tokenade logs,
  caches and folds.
• Nothing is lost — every fold carries a hash; tokenade expand-ref
  gives the exact bytes back.

Install

npm (prebuilt binaries)

npm install -g @tokenade/cli
tokenade install      # detects your coding agents and wires Tokenade in
tokenade healthcheck  # optional: verify the wiring

The binary for your platform ships as an optional per-platform npm package;
if that is skipped, postinstall downloads the archive from this repository's
GitHub Release and checks it against the release's Ed25519-signed manifest
(SHA-256). Linux (x64 glibc/musl, arm64), macOS (x64, arm64), Windows (x64,
arm64). Node ≥ 18.

Prebuilt archives are also attached to every
GitHub Release.

Build from source

Requires a recent stable Rust toolchain (rustup) and a
C/C++ compiler (tree-sitter grammars and SQLite are compiled from source).

git clone https://github.com/pi-infected/tokenade
cd tokenade
cargo install --locked --path crates/tokenade-cli   # or use a script below
tokenade install

Per-OS scripts in scripts/build/ build the same targets as the release CI
and print where the binary is:

• Linux (x86_64 / aarch64): scripts/build/build-linux.sh for a native
  glibc build; --musl for a static binary (uses
  cargo zigbuild if
  installed); --target <triple> for anything else. lld is used as linker
  on x86_64 when present.
• macOS (Xcode Command Line Tools): scripts/build/build-macos.sh for
  this Mac's architecture; --target aarch64-apple-darwin (Apple Silicon) or
  --target x86_64-apple-darwin (Intel); --universal for both, merged with
  lipo.
• Windows (PowerShell): .\scripts\build\build-windows.ps1 builds
  x86_64-pc-windows-gnu with a static CRT; it needs MinGW-w64 (gcc, g++)
  on PATH, e.g. from MSYS2. -Target aarch64-pc-windows-gnullvm builds
  ARM64 with llvm-mingw.

The semantic-search embedding model is downloaded on first use (Hugging Face,
with a signed GitHub Release asset as fallback) and cached; set
TOKENADE_MODEL_DIR to a directory that already holds it for offline machines.

Supported agents

Full integration (hooks: command compaction, read dedup, web, MCP, code
search, lean output, context optimization): Claude Code, OpenCode,
Kilo Code, Copilot CLI, Hermes, Cline, T3 Code.

Most features: OpenClaw and Pi (no MCP wrapping); Qwen Code (no
web replacement); Codex CLI and the Codex app, Cursor, Grok (no
web replacement or read dedup); Droid (no web, MCP wrapping or read
dedup).

Partial: Aider (run commands through tokenade wrap …),
Antigravity and Claude Desktop chat (MCP optimization only).
Claude Desktop's Code tab runs Claude Code and gets full integration.

tokenade install --only <agent> limits the install to one agent;
tokenade uninstall removes everything it added.

Network access

Your savings
ledger and dashboard (tokenade gain, tokenade dashboard) stay on your
machine. The only network calls are:

• the ones you ask for (tokenade web, tokenade search);
• the one-time embedding-model download (see above);
• a daily auto-update check that reads the signed manifest.json of the
  latest GitHub Release and only installs a binary whose signature and
  SHA-256 verify.

Turn auto-update off with TOKENADE_NO_AUTO_UPDATE=1 (or
tokenade upgrade --off); tokenade upgrade updates on demand.
TOKENADE_NO_WEBSEARCH=1 / TOKENADE_NO_WEBFETCH=1 keep your agent's own
web tools.

tokenade install ends with a one-time line inviting you to star the
repository; TOKENADE_NO_STAR_INVITE=1 turns it off.

Upgrading from 1.x

1.x installs update themselves to 2.0 (or run npm i -g @tokenade/cli@latest).
On its first run, 2.0 deletes the leftover 1.x licence, quota and
usage-report files under ~/.tokenade once and keeps your savings history.
tokenade login, activate and report no longer exist. Run
tokenade install again if your agent's hooks look stale.

Everyday commands

tokenade wrap 'cargo test'          # run a noisy command, get the compacted output
some-cmd | tokenade read -          # compact any blob; format auto-detected
tokenade map                        # codebase structure
tokenade query parse_config         # locate a symbol
tokenade skeleton src/lib.rs        # signatures only
tokenade semantic "where is auth"   # search by meaning
tokenade expand-ref <hash>          # get folded bytes back
tokenade dashboard                  # what you saved: graphical local report
tokenade gain                       # same totals, in the terminal
tokenade help                       # everything else

Contributing

Bug reports, compactor fixtures and pull requests are welcome — see
CONTRIBUTING.md and the
Code of Conduct. Security issues: see
SECURITY.md.

License

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE)
• MIT license (LICENSE-MIT)

at your option (MIT OR Apache-2.0).

Unless you explicitly state otherwise, any contribution intentionally
submitted for inclusion in this project by you, as defined in the Apache-2.0
license, shall be dual licensed as above, without any additional terms or
conditions.