llmgps

中文文档

llmgps is a self-hosted multi-model reasoning workspace. It gives several LLMs the same
question and the same retrieved evidence, collects their independent answers, optionally lets
them debate the points they disagree on, and synthesizes one final response.

It started from a practical annoyance: Claude, ChatGPT, DeepSeek and friends each have real
strengths, but using them in separate tabs means duplicated context, inconsistent research, and
manually reconciling three different answers yourself. llmgps does that reconciliation in one
place, on your own machine, with your own keys.

Why this is not just "chat with multiple models"

• Shared evidence. Web retrieval happens once, centrally (Brave or Tavily), and the same
  evidence block is handed to every responder — so models disagree about reasoning, not about
  which search results they happened to get.
• Independent opinions first. Responders never see each other's answers in the first round.
  Cross-contamination is deliberately delayed until the debate phase.
• Deliberation is gated, not unconditional. A synthesizer model is asked whether the
  responders already broadly agree. Debate rounds only run when they don't.
• Failures degrade instead of aborting. A responder that errors is recorded as a failure and
  the run continues on the survivors. Search failures, compression failures and memory-relevance
  failures are all non-fatal.
• Expensive work is not thrown away. If only the final synthesis fails, you get the raw model
  opinions back and can retry synthesis alone — without re-running the fan-out or the debate.
• Local-first configuration. Provider keys, settings and chat history live in one
  AES-256-GCM-encrypted row in a local SQLite file. Nothing is sent to a llmgps-operated service,
  because there isn't one.

llmgps does not claim to have invented multi-agent debate, LLM councils, or mixture-of-agents.
It is a practical, self-hostable implementation of those ideas that you can run and modify.

Looking for contributors

This is a small project with a lot of surface area, and it is actively looking for collaborators.
Concretely, these are the areas where help would matter most right now:

Start with CONTRIBUTING.md for setup, architecture orientation and the
review expectations, and docs/ARCHITECTURE.md if you want the deeper
technical picture first. If you use a coding agent, point it at AGENTS.md.

Small, focused pull requests are very welcome. So are issues that just describe a problem you hit.

The three modes

Debate mode is the expensive one: with 5 responders and 2 rounds you are paying for up to 15
model calls plus consensus checks and synthesis. GPS mode is the common case.

Debate loop in detail

1. Fan-out. All responders receive the same prompt (plus shared evidence, if search is on)
   and answer independently, in parallel.
2. Consensus check. The synthesizer model is shown all opinions and answers a single
   YES/NO: do these broadly agree on the core answer? If the check itself fails, llmgps
   conservatively assumes no consensus.
3. Debate round. On NO, every model is shown the other models' answers and asked to
   address specific disagreements point-by-point.
4. Evidence refresh. Before round 2 — and only if web search is enabled and configured —
   retrieval runs again so the second round argues over fresher evidence. A failed refresh is
   non-fatal; the round continues with the previous evidence.
5. Synthesis. After consensus is reached or 2 rounds are exhausted, the synthesizer produces
   the final answer. Optionally, the debate is compressed into a structured summary first to keep
   the synthesis prompt small.

Note on wording: llmgps produces a web-grounded, multi-model synthesized answer. Retrieved
search snippets are evidence supplied to the models, not verification — the final answer is not
fact-checked by the system.

Shared retrieval architecture

When web search is enabled, retrieval happens once per round and is shared by every responder:

user question
  → search-query model generates up to 5 focused queries   (optional; falls back to the raw question)
  → Brave or Tavily is queried per query
  → results deduplicated by URL, formatted into one dated evidence block
  → the block is prepended to the conversation as a user/assistant pair
  → every responder model receives the identical evidence
  → consensus check → debate rounds (with optional evidence refresh)
  → synthesis

Screenshots

Features

• Three reasoning modes: Single, GPS and Debate, switchable per message.
• 12 provider integrations: OpenAI, Anthropic, Gemini, OpenRouter, DeepSeek, Kimi, Qwen,
  Mistral, Zhipu, xAI, any OpenAI-compatible custom endpoint, and Ollama — behind four request
  shapes (openai-compatible, anthropic, gemini, ollama).
• You choose the models: model IDs are entered per provider in Settings rather than picked
  from a hard-coded list, so new model releases work without waiting for a llmgps update.
• Live streaming: token-level output and reasoning/"thinking" traces stream from every
  responder at once (OpenAI-compatible, Anthropic and Ollama stream incrementally; Gemini is
  buffered).
• Ollama integration: local models as first-class responders or synthesizers, with model
  discovery and an optional proxy bypass.
• Web search: Brave or Tavily, with an optional dedicated model for query generation.
• Context compression & rolling context: compress a long debate into a structured summary,
  and optionally carry that summary forward as the context for the next turn.
• User memories: up to 20 short notes, injected only when a relevance check says they are
  useful for the current question.
• Extended thinking: opt-in Anthropic extended thinking with a configurable token budget.
• Encrypted local storage: settings, provider keys and chat history in an AES-256-GCM
  encrypted payload inside SQLite.
• Proxy support: HTTP or SOCKS5 for provider and search traffic, with an Ollama bypass.
• Bilingual UI: English and Simplified Chinese.

Deployment & Setup

Docker (Recommended)

Pull and run:

docker pull ghcr.io/leodenglovescode/llmgps:main

docker run -d \
  -p 3000:3000 \
  -e LLMGPS_DATA_KEY=your-secret-key \
  -v llmgps-data:/data \
  --name llmgps \
  ghcr.io/leodenglovescode/llmgps:main

Access: http://localhost:3000

Set LLMGPS_DATA_KEY to any strong secret string. The /data volume persists your history and settings.
Containerized runs require LLMGPS_DATA_KEY or LLMGPS_DATA_KEY_FILE — the app refuses to
generate and persist a key next to the database in Docker.

Local Development

Requires Node.js 22 (the app uses the built-in node:sqlite module; CI and the Docker image
both run Node 22).

1. Install: npm install
2. Run: npm run dev
3. Access: http://localhost:3000

On first run you create a single owner account (username + password) in the browser, then add
provider API keys and model IDs in Settings.

Other scripts: npm run build, npm run start, npm run lint. Type-check with npx tsc --noEmit.
There is no test script yet — see CONTRIBUTING.md.

Environment variables

See .env.example. All are optional in local development:

Data & Security

llmgps stores all sensitive configuration in an encrypted payload inside SQLite.

• Encryption: AES-256-GCM, key derived with scrypt from LLMGPS_DATA_KEY.
• Key handling: in local development a .llmgps-data.key file is generated automatically
  (mode 0600). In Docker you must supply the key yourself.
• Persistence: mount /data to keep your history and settings across container restarts.
• Single owner: one account per deployment, session cookie signed with an HMAC secret that
  itself lives in the encrypted payload.

This protects data at rest against someone reading the database file without the key. It is not a
substitute for not exposing the app to the public internet.

Ollama Support

1. Start Ollama.
2. In llmgps Settings, enable Ollama and set the base URL (e.g. http://127.0.0.1:11434).
3. Add the local model IDs you want and select them as responders or as the synthesizer.

If you route provider traffic through a proxy, enable bypass proxy for Ollama so local
requests go direct.

Debate Mode Flow Chart

Documentation

• CONTRIBUTING.md — setup, architecture orientation, checklists, PR expectations
• docs/ARCHITECTURE.md — how the system actually works, file by file
• AGENTS.md — short entry point for coding agents

Credits & License

• Logic & Idea: @leodenglovescode
• Assistance: GitHub Copilot

Licensed under the MIT License.