RustXNG

A self-hosted, SSRF-hardened web search and grounding engine for local AI agents.

No API keys. No fees. One request returns ranked results and the page text, so an agent can answer from sources instead of guessing.

日本語

Why this exists

An agent searches, reads, and decides — then does it again. What it needs underneath is not another search UI but a retrieval layer:

What an agent needs	What RustXNG does
No keys, no per-seat billing	Free, keyless, self-hosted
Stay inside your network	Runs on your machine; search terms never leave it
Fast enough to call 50 times a task	0.01 s on a repeated query (cached)
Text to ground on	~32,000 characters from the top 5 pages in one request
Survive one dead source	180 engines in parallel, per-engine suspension and rest
Safe with untrusted URLs	Public addresses only (SSRF guard)

Install

1. Prebuilt binary (fastest)

Download the archive for your platform from Releases and put rustxng on your PATH.

curl -fsSL -o rustxng \
  https://github.com/sahenjp/RustXNG/releases/latest/download/rustxng-v0.25.0-x86_64-unknown-linux-musl
chmod +x rustxng && ./rustxng --version

Check the .sha256 published next to each asset before you run it.

2. Cargo

cargo install --git https://github.com/sahenjp/RustXNG.git --tag v0.25.0

This installs two binaries: rustxng (the server) and rustxng-index (the optional local index).

3. Docker

docker run -p 8888:8888 ghcr.io/sahenjp/rustxng:latest

Published from v0.26.0 onwards. To build it yourself instead:

git clone https://github.com/sahenjp/RustXNG.git && cd RustXNG
docker build -t rustxng . && docker run -p 8888:8888 rustxng

Bind your own settings with -v "$PWD/settings.toml:/app/settings.toml:ro".

4. From source

git clone https://github.com/sahenjp/RustXNG.git
cd RustXNG
cargo build --release
./target/release/rustxng

Requires Rust 1.85 or newer. No system libraries beyond a C toolchain.

Quickstart

# start on 127.0.0.1:8888
rustxng &

# search
curl "http://127.0.0.1:8888/search?q=tokio+runtime&format=json"

# search + page text for grounding
curl "http://127.0.0.1:8888/search?q=tokio+runtime&format=json&include_raw_content=1"

Response shape (SearXNG compatible):

{"results": [{"title": "...", "url": "...", "content": "...", "raw_content": "..."}]}

The browser UI is at http://127.0.0.1:8888/.

How it compares

100 questions with known answer pages, measured on the same machine:

	RustXNG	Exa	Tavily
hit@1 (right page first)	79–83	77	34
hit@5	92–96	91	79
MRR	0.853–0.889	0.840	0.514
Cost	$0	paid	$8/1k
Page text per request	~32,000 chars	~9,000	~1,200
Repeat query	0.01 s	0.6 s	~1 s
Cold query	0.8–1.6 s	0.6 s	~1 s

Two caveats, stated plainly: the numbers move by a few points between runs, and Google is unavailable in this build (its WML endpoint now returns 403, the HTML endpoint requires JavaScript, and headless browsers are detected). See docs/COMPARISON-EXA.md for a plain-language version.

What you get

226 engine types ported from SearXNG, 352 settings entries, 180 enabled by default
Ranking that beats raw engine order: term coverage, canonical-site authority, identifier match (RFC numbers), thumbnail-free deduplication
Resilience: per-engine timeouts, suspension with backoff, and automatic rest that drops engines which keep answering nothing
Optional local index (rustxng-index): crawl your own corpus, rank with BM25, merge into results — experimental, off by default
Ops: /healthz, OpenMetrics /metrics, response cache, image/favicon proxy, bot rate limiting, RUSTXNG_IDLE_TIMEOUT for sidecars

Documentation


docs/AI-INFRA.md	Why this is agent infrastructure, and why tests and audits matter
docs/COMPARISON-EXA.md	The comparison above, explained without jargon
docs/BENCHMARK.md	How the numbers were produced, including the variance
docs/COVERAGE.md	Every engine, mapped to its SearXNG counterpart
docs/COMPATIBILITY.md	Where this matches SearXNG and where it differs
docs/LOCAL-INDEX.md	The experimental local index (crawl + BM25)
docs/PORTING.md	Internals and configuration
SECURITY.md	Vulnerability scan results and known advisories

Contributing

Issues and pull requests are welcome — see CONTRIBUTING.md. The rules are short: run cargo fmt, cargo clippy, cargo test, and keep the nightly benchmark green.

Good first issues live in the tracker; the parsers under src/engines/ are deliberately small and each has a fixture test.

License

AGPL-3.0-or-later. This is a port of SearXNG; attribution and the exact scope of the port are in NOTICE. If you run a modified version as a network service, AGPL section 13 asks you to offer its source to your users.