Build Your Own WIKI

Turn the web, your files, and loose notes into a personal wiki that an LLM agent
keeps coherent: search and summarize pages, ingest them into a contextual vector
index for grounded, cited Q&A, and grow a layered knowledge graph that de-duplicates
entities, builds topics, and writes synthesis pages. One Flask app — web UI, JSON API,
and a runner.py CLI. Inspired by LangChain's llm-wiki deep-agents example, but it
builds a private, on-disk library instead of syncing to a hub. Local-first: every
store is plain JSON/SQLite under data/; cloud is opt-in.

Features

• Agentic knowledge graph — a deepagents
  agent (local filesystem backend, no cloud sandbox) saves passages, extracts entities
  and typed relations, canonicalizes duplicates, nests topics, and writes synthesis pages.
• Contextual RAG — Anthropic-style contextual retrieval over a two-layer HNSW index
  (section summaries + chunks), with an LLM re-ranker (precision) or document-aware MMR
  (multi-doc recall). Answers are grounded and cited.
• Memory — a cross-session store recalled before every answer and written back
  after (observations, 👍/👎, corrections that supersede stale notes); it improves from
  use, not just ingestion.
• Agent skills — turn selected context into a reusable, evaluated skill via a
  sub-agent pipeline (understand → analyze → author → eval → gate → refine). Authored by
  the latest Claude in-process or via the Claude Code CLI as a subprocess, scored by a
  deterministic + rubric panel, and gated behind human review before it joins the
  library — optionally as a durable LangGraph build that pauses at the gate and resumes later.
• MCP, both directions — connect agents to external MCP servers (Supabase, GitHub, …)
  and run BYO-WIKI as an MCP server. Reads join the agent; writes are deny-by-default.

Architecture

flowchart TB
  subgraph IF["Interfaces"]
    direction LR
    UI["Web UI · Flask"]
    API["JSON API · /api"]
    CLI["CLI · runner.py"]
  end

  ING["Ingestion pipeline<br/>fetch · clean · contextual summaries<br/>chunk · embed · entity / relation / topic extraction"]

  subgraph STORE["Stores · local-first under data/"]
    direction LR
    KG[("Knowledge graph · layers 0–5<br/>source · section · chunk<br/>entity · topic · synthesis")]
    VEC[("HNSW index<br/>section summaries + chunks")]
    MEM[("Memory · layer 6")]
    SKL[("Skill library · layer 7")]
  end

  subgraph REASON["Reasoning"]
    direction LR
    RAG["Contextual RAG<br/>retrieve → re-rank / MMR → cite"]
    AGENT["Curating agent · deepagents<br/>ingest · query · lint"]
    SKILL["Agent-skill loop · LangGraph<br/>author → eval → gate → human review"]
  end

  IF --> ING
  IF --> REASON
  ING --> KG
  ING --> VEC

  VEC --> RAG
  RAG --> EMB["Embeddings · OpenAI"]
  RAG <--> MEM
  KG <--> AGENT
  AGENT <--> MEM
  AGENT <--> MCP["External MCP servers<br/>Supabase · GitHub · fetch<br/>reads join agent · writes gated"]
  SKILL --> SKL
  SKILL --> KG
  SKILL --> MCP

  REASON --> LLM["LLM providers<br/>Claude · OpenAI · Qwen<br/>DeepSeek · Gemini · Mistral"]

  REASON -.->|offline metrics| EVAL["Evaluation<br/>hit@k · MRR · recall<br/>RAGAS · 5-family judge panel"]
  REASON -.->|live run traces| OBS["Observability<br/>LangSmith · OpenTelemetry"]

Everything above the stores is stateless; all state lives in data/ (JSON graph,
HNSW vectors, SQLite checkpoints), so the app is reproducible and local-first. The JSON
API can also be exposed as an MCP server (reads open, writes deny-by-default).

Evaluation and observability are different planes — don't conflate them. Evaluation
measures quality offline: deterministic retrieval metrics (hit@k · MRR · recall),
RAGAS (faithfulness · answer-relevancy · context-precision, wrapped as LangSmith
evaluators in ragas_eval.py), and a 5-family LLM-judge panel calibrated against human
labels. Observability traces live runs — agent, RAG, and skill builds — to
LangSmith and, optionally, OpenTelemetry. RAGAS is an evaluator, not a tracer.

How the curating agent works

A deepagents agent drives four modes
(init · ingest · query · lint) as a recursion-limited reason↔act loop over a local
filesystem backend. Its tools read and write the knowledge graph, recall memory before
an answer and write it back after, look up skills, and call external MCP read tools.
query grounds cited answers through contextual RAG; ingest/lint canonicalize
duplicates, nest topics, and write synthesis pages.

flowchart TB
  subgraph DRIVE["Entry points · Web UI · /api · runner.py"]
    direction LR
    INIT["init<br/>scaffold"]
    INGEST["ingest<br/>organize new material"]
    QUERY["query<br/>cited answer"]
    LINT["lint<br/>maintain library"]
  end

  DRIVE --> REASON

  subgraph LOOP["deepagents · local filesystem backend"]
    REASON["reason ↔ act loop<br/>(recursion-limited)"]
    KGRW["KG tools<br/>search · upsert · merge<br/>relate · nest topics · synthesize"]
    MEMT["Memory tools<br/>recall before · write after"]
    SKT["Skill lookup"]
    MCPR["MCP read tools"]
    REASON <--> KGRW
    REASON <--> MEMT
    REASON <--> SKT
    REASON <--> MCPR
  end

  KGRW <--> KG[("Knowledge graph<br/>layers 0–5")]
  MEMT <--> MEM[("Memory · layer 6")]
  QUERY -.->|grounds answers| RAG["Contextual RAG<br/>retrieve → re-rank / MMR"]
  RAG --> VEC[("HNSW index")]
  RAG --> KG

Skill builds run as a durable LangGraph

The skill loop has two interchangeable runtimes over the same phase functions: a linear
in-process pipeline (SKILL_BACKEND=pipeline), or a LangGraph StateGraph
(skill_graph.py) that adds conditional gating, a durable human-review interrupt(), and
checkpointing (SQLite / Postgres / memory). Each phase is authored either in-process or by
the Claude Code CLI as a subprocess (generate_cc). The gate never finalizes alone —
it lands a draft in pending_review; only a human promotes it. Pause at the gate,
checkpoint, and resume in another process by thread_id. Every build/eval/refine is traced
to LangSmith (parent run + per-phase children) and, optionally, OpenTelemetry.

stateDiagram-v2
  direction TB
  [*] --> gather
  gather --> understand: backend = pipeline
  gather --> generate_cc: backend = claude_code
  understand --> analyze
  analyze --> codeact
  codeact --> evaluate
  generate_cc --> evaluate
  evaluate --> human_review: pass gate, pending_review
  evaluate --> [*]: reject
  human_review --> finalize: accept
  human_review --> prepare_revision: revise + notes
  human_review --> [*]: reject
  prepare_revision --> codeact: pipeline
  prepare_revision --> generate_cc: claude_code
  finalize --> [*]

  note right of evaluate
    deterministic checks (11)
    + 5-family rubric panel
  end note
  note right of human_review
    interrupt() pauses + checkpoints
    (SQLite / Postgres / memory)
    resume by thread_id, survives restart
  end note

Quick start

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt          # add `ragas` for RAGAS metrics
cp .env.example .env                      # set at least one provider key
python app.py                             # → http://localhost:5000

Both app.py and runner.py auto-load .env. RAG needs OPENAI_API_KEY for
embeddings (text-embedding-3-small) regardless of chat provider — Anthropic has no
embeddings API.

How it works (the critical bits)

The graph is layered, not a flat entity bag. Nodes climb source → section → chunk → entity → topic → synthesis → memory → agent_skill (layers 0–7), joined by typed edges
(mentions, relation, belongs_to, subtopic_of, …). The KG tab renders three
granularities (document / section / chunk).

Retrieval is vector-only — on purpose. Each section gets one LLM-written contextual
summary; chunks are embedded with title · date · section prepended into a two-layer
HNSW index. Retrieval ranks sections, drills into chunks, then re-ranker or MMR picks
top-k. Entity-graph traversal was built, evaluated, and removed — MMR beat it on
recall and judged synthesis. The graph earns its keep in construction
(canonicalization, topics, provenance) and presentation (concept maps), not retrieval.

Skills are gated. A drafted skill must clear deterministic checks (usable
description, ≥2 steps, positive and negative triggers, declared tools, no placeholders)
and a cross-family rubric judge panel — but the gate never finalizes alone: it lands in
pending_review and only a human promotes it. Every build/eval/refine is logged with
per-phase timings, tokens, and triggering precision/recall; refine rebuilds a skill from
its measured weaknesses. Optionally runs as a LangGraph StateGraph where human review is
a durable interrupt() — pause at the gate, checkpoint, resume in another process.

Evaluation

All numbers come from a version-controlled 28-doc corpus (eval/corpus_urls.txt, LangChain

• Anthropic articles). Deterministic retrieval metrics are the most comparable across runs;
  absolute LLM-judge scores are judge-dependent, so judging uses a 5-family panel
  (gpt-5.2 · qwen3.7-plus · deepseek-v4-flash · gemini-3.5-flash · mistral-large-2512) and
  reports judge_alignment against human labels.

MMR winning on recall and synthesis is why graph-RAG was dropped; the relevancy jump
came from letting the generator synthesize across passages instead of refusing when no
single chunk states the connection.

Interfaces

Web UI — tabs for Read (search / fetch+extract / summarize / save-to-KG), Knowledge
Graph (ingest, browse the layer-colored graph, Integrate / Ask / Maintain), Library Q&A
(cited answers, re-ranker/MMR toggles), Memory, and Agent Skills (build, watch the
eval/gate report, accept/revise/reject; toggle durable ⛓ for a checkpointed LangGraph build).

HTTP API — under /api: read · kg · agent · memory · rag · skill · mcp.

CLI (runner.py --mode …):

init · ingest · query · lint                     # knowledge-graph workspace
memory-add · memory-recall · memory-list         # cross-session memory
skill-build · skill-pending · skill-review · skill-refine · skill-observability
skill-graph-build · skill-graph-resume           # durable, pause-at-review build
rag-ingest · rag-ask · rag-eval · rag-experiment · rag-ragas · rag-crossdoc
mcp-list · mcp-ingest · mcp-call · mcp-serve      # MCP client + serve-as-server

Common flags: --provider, --model, --rerank/--no-rerank, --mmr, --no-agent.

Screenshots

The web UI in action — one image per tab.

Read — search / fetch + extract / summarize / save-to-KG.

Cache — reuse extracted content across the KG, Q&A and Memory without re-fetching.

Knowledge Graph — ingest, browse the layer-colored graph, Integrate / Ask / Maintain.

Memory — cross-session store recalled before every answer and written back after.

Agent Skills — build, watch the eval/gate report, accept/revise/reject.

Connecting tools (MCP)

Agents can call external MCP servers, and BYO-WIKI can run as one — both opt-in and
local-first. Enable servers with MCP_ENABLED; read tools join the curating agent and the
skill-builder, and mcp-ingest stages a read tool's output into the KG. Writes are
deny-by-default: a write runs only via /api/mcp/write (or mcp-call --confirm) with
MCP_ALLOW_WRITES=1 and explicit human approval, and the client gate tracks the server's
own read_only scope so the two never drift. The hosted Supabase MCP is HTTPS, so it works
even where direct Postgres is blocked.

pip install langchain-mcp-adapters mcp
export MCP_ENABLED=supabase SUPABASE_ACCESS_TOKEN=… SUPABASE_PROJECT_REF=…
python runner.py --mode mcp-list

See docs/mcp-proposal.md for the full design.

Configuration

Provider keys live in .env (gitignored — never commit real keys); see .env.example for
the complete list. Common extras:

Project layout

app.py / runner.py                     Flask app (UI + API) / CLI
agent.py kg_tools.py                   deepagents harness + graph tools
knowledge_graph.py                     multi-layer graph store + queries
ingestion.py extraction.py enrich.py   chunking · entity/relation/topic extraction
pipeline.py embeddings.py vectorstore.py   contextual ingest + HNSW + MMR
rag.py rag_experiment.py ragas_eval.py crossdoc.py   retrieval, answers, eval suite
memory.py memory_tools.py              cross-session memory (layer 6)
skill_*.py                             agent-skill build loop, eval/gate, observability, LangGraph (layer 7)
mcp_config.py mcp_tools.py mcp_server.py   MCP client (gated) + BYO-WIKI as an MCP server
providers.py config.py                 provider table + judge selection; .env / LangSmith
eval/ static/ templates/ data/         corpus + datasets; UI; local stores

Credits

Inspired by LangChain's llm-wiki deep-agents example. Built with Flask,
deepagents/LangChain, hnswlib, OpenAI embeddings, RAGAS, and LangSmith.