mcp-toolsearch-proxy

mcp-toolsearch-proxy lets a model use hundreds or thousands of MCP tools without loading them. On 500 tools, each
request carries 97% fewer tokens of tool definitions, a task costs about half as much, and the model reaches the right
tool nearly as often as with every tool loaded (benchmarks).

It is an MCP server that puts six tools in front of an MCP server with far too many
to list to a model, such as a gateway that aggregates dozens of servers:

A catalog of a thousand tools costs a model over 100,000 tokens of tool definitions before it does anything, and many
clients cap how many tools they load anyway. Behind the proxy, the model sees six, in about 2,000 tokens.

A call whose arguments do not fit the tool's input schema (a required property missing, a wrong type, a value outside
an enum) is refused before it reaches the upstream, with what is wrong and the schema, so the model can fix it at once.
The same goes for an unknown name that is clearly a typo of one tool, and for an upstream that answers invalid params.

Calls come in separate tools by what the upstream tool may do, so that a client can tell them apart and ask for each
its own way. call_read_tool and call_read_tools are marked read-only and can be allowed to run without asking.
call_write_tool is marked as a write that destroys nothing and may be allowed too, or keep asking. call_destructive_tool
is marked destructive and can keep asking every time. Writes are never batched, so each keeps its own approval.

The proxy reads all four of the upstream's tool annotations, with the spec's defaults for any a server leaves out. A
tool is a read only when marked readOnlyHint, and a write that only adds only when also marked destructiveHint: false, since the spec's default is true; anything else, unmarked tools included, is destructive, and so are any the
operator lists in -write-tools. search_tools and describe_tool show each tool's readOnly, destructive,
idempotent and openWorld, and a write that got no result says whether its server marks it idempotent, so whether
calling it again is safe on that server's word. The annotations choose the call tool, and with it the client's
approval; they are the upstream's word, not a security boundary. Allowing call_read_tool allows every read-only tool
of every server behind the upstream, including servers added later; docs/clients.md says what each
grant covers and how to set up claude.ai and Claude Code.

client ──▶ gateway /mcp/search ──▶ mcp-toolsearch-proxy ──▶ gateway /mcp/all ──▶ MCP servers

How it works

The proxy has no credentials. Each request it sends upstream carries the caller's Authorization header and nothing
else, so the upstream server (usually the same gateway, on another route) authenticates and authorizes the caller as
if they had come directly. Whatever the caller isn't allowed to see or call, the proxy can't see or call either.

Each caller gets their own catalog: the tools the upstream lists for that caller. It is kept for ten minutes, then
listed again in the background while the old one is still used, and it is looked up by a hash of the token. A claim
inside the token, such as sub, would be easier to key on but can't be trusted without verifying the token, so the
proxy doesn't use one.

Search combines two rankings. BM25 matches words in tool names, descriptions and parameters, with Lithuanian and
English endings stemmed and diacritics ignored. A multilingual embedding model,
potion-multilingual-128M (MIT), matches by meaning. The
model runs inside the proxy, in pure Go, without a GPU or an embedding service, and embedding a query takes
microseconds. Without the model, search falls back to BM25 alone, which matches only words a tool's text shares with
the query: it stems no endings but Lithuanian and English ones, and does not split Chinese or Japanese text into words.

The upstream server's own instructions still reach the model. The proxy answers each caller's handshake with its own
short guidance followed by the upstream's instructions, unchanged, as the caller's catalog received them, up to
-max-instructions characters (10,000 by default; longer ones are cut, with a note saying so). Clients cut it again,
much sooner: Claude Code keeps about 4,000 characters of a server's instructions, so only the start of a long upstream
text reaches the model (see docs/clients.md). And search_tools
returns the input schemas of its best few matches (-search-schemas, 3 by default), so a call usually follows a
search directly.

Logs and metrics record tool names, outcomes and timings. Queries, arguments, results and tokens never appear in
them; docs/privacy.md has the details. The transport is streamable HTTP without sessions, for
MCP 2026-07-28 clients and older ones, so a restart or a load balancer doesn't break a client's connection.
-transport stdio serves the same tools on standard input and output instead, for clients that start servers as local
commands. There is no caller's token there, and the proxy has none of its own, so stdio works only with an upstream
that needs no authentication (docs/clients.md).

It works with any upstream that speaks streamable HTTP MCP and authorizes by bearer token.
docs/agentgateway.md shows a complete agentgateway setup, which
the end-to-end tests run.

Status

Early development. The tools, their arguments and the settings may still change before 1.0.

Install

Homebrew, on macOS or Linux:

brew install --cask datolabs-io/tap/mcp-toolsearch-proxy

Packages (.deb, .rpm), archives and a container image are attached to each
release. See docs/deploy.md.

mcp-toolsearch-proxy fetch-model -model-dir ./model        # once: ~530 MB, checked against pinned digests
mcp-toolsearch-proxy -upstream https://gateway.example.com/mcp/all -model-dir ./model

-embedder none skips the model and searches lexically only. docs/configuration.md lists
every setting, as mcp-toolsearch-proxy -h does; each has an environment variable too. Then connect a client:
docs/clients.md.

Clients that already defer MCP tools and search them on demand, as Claude Code does, gain less from the proxy; see
when you don't need this.

Search quality

The repository holds an evaluation set of made-up tools, described in English, and questions people ask about them
in English and Lithuanian. Recall at 5 today:

Words alone can't connect a Lithuanian question to an English description. The model can. Tests hold changes to
ranking to these numbers.

Benchmarks

On a made-up catalog of 500 tools, a model given 30 questions reaches the right tool nearly as often through the proxy
(90%) as with every tool loaded (95%), at about half the cost per task, and more often than with Anthropic's tool
search (87%). docs/benchmarks.md has the numbers, what a client loads in each mode at up to 1,490
tools, and how to run the benchmarks.

Development

mise install          # the pinned toolchain and the git hooks
mise run ci           # lint, scan and unit tests, as CI runs them
mise run test:model   # the built-in model against its reference implementation and the evaluation set
mise run e2e:up       # agentgateway, the proxy and fixture MCP servers, with Docker Compose
mise run test:e2e     # the end-to-end tests, against that stack or any at E2E_SEARCH_URL and E2E_ALL_URL
mise run docs:config  # regenerate docs/configuration.md after changing a flag

See CONTRIBUTING.md and ARCHITECTURE.md.

Licence

MIT. The built-in model is published by Minish under the MIT licence and is downloaded, not distributed,
by this project.