The Distillery

Cut your input tokens before they reach the model.

The Distillery is a local token-optimization proxy for AI coding tools. It sits between your coding agent and the model API, removes redundant input context, and reports the overall input reduction across your sessions. It is built for developers who want lower model bills without prompt rewriting, workflow changes, or sending code to The Distillery's hosted service for optimization.

Install

npm install -g thedistillery

Requires Node.js >=20.9.0 <25.

Quick Start

# Log in once
thedistillery auth login

# Start the local proxy
thedistillery start

# Use Claude Code normally, then inspect savings
thedistillery stats

thedistillery start binds to 127.0.0.1:3080 by default and automatically configures Claude Code routing. For other clients, point the tool at the local Anthropic or OpenAI-compatible endpoint shown below.

What It Does

The Distillery optimizes request input before it reaches the upstream model API:

1. Deduplication - removes identical tool outputs already seen in the session.
2. Tool output distillation - trims oversized shell, search, diff, and file outputs while preserving useful head/tail context.
3. Retention - keeps the most useful session context when history grows too large.
4. Prompt-cache placement - on paid plans, places cache breakpoints automatically where supported.

The displayed reduction is an input-token metric. Output tokens are streamed back unchanged.

Supported Clients

The Distillery works with Claude Code automatically and with any client that supports a custom Anthropic base URL or OpenAI-compatible base URL.

Full client guides live at thedistillery.dev/clients. Codex CLI must use API-key auth; ChatGPT login uses a websocket backend and bypasses OPENAI_BASE_URL.

Using AWS Bedrock through Bedrock-Mantle? See Bedrock setup.

Configuration

Create .distilleryrc.json in your project directory, or any parent directory, to set a project-level preset:

{ "preset": "smart" }

Legacy preset names still work: conservative maps to light, balanced maps to smart, and aggressive maps to deep.

Disable compression for a project:

{ "preset": "smart", "compression": false }

Environment variables:

CLI

thedistillery start              # Start the local proxy
thedistillery start --port 4000  # Use a different port
thedistillery stop               # Stop the proxy
thedistillery stats              # Show cumulative token and cost savings
thedistillery preset smart       # Switch global preset
thedistillery config show        # Show resolved config
thedistillery sessions --detail  # Show recent sessions, Pro only
thedistillery export --format csv --output sessions.csv
thedistillery billing status
thedistillery billing breakdown --month 2026-07
thedistillery billing manage
thedistillery uninstall --dry-run

For one-off commands, run a child process with an embedded proxy:

thedistillery run claude --watch

Data And Privacy

Optimization runs locally on your machine. The hosted service receives usage metadata needed for account, billing, dashboard, alerts, and attribution features: token counts, cost deltas, model/session metadata, and project attribution. It does not receive your code or prompts for optimization.

Local session data is stored in:

~/.distillery/data.db

Self-Hosting And Safety

The proxy is designed for loopback-only use and binds to 127.0.0.1 by default.

Do not bind it to 0.0.0.0 unless you understand the risk. Binding to all interfaces can expose a proxy that forwards requests using your API credentials.

Supported local deployment:

ANTHROPIC_BASE_URL=http://127.0.0.1:3080

Uninstall

Interactive teardown:

thedistillery uninstall

Preview what would be removed:

thedistillery uninstall --dry-run

Full uninstall guidance: thedistillery.dev/docs/uninstalling.

Development

npm install
npm test
npm run build:cli

Useful scripts:

npm run dev                 # Run the CLI from source
npm run benchmark:tokens    # Compare token counts
npm run generate:benchmark  # Regenerate benchmark JSON
npm run build               # Build marketing, dashboard, API, and CLI

Links

• Website: thedistillery.dev
• App: app.thedistillery.dev
• API: api.thedistillery.dev
• Install guide: thedistillery.dev/install
• Client guides: thedistillery.dev/clients
• Agent setup guides: thedistillery.dev/for-agents

License

This project is licensed under the Elastic License 2.0.