LLM Wiki v2

Give it your documents. It files them into a tidy wiki of Markdown notes and
answers your questions, citing the notes it used. It runs on your computer,
with any AI provider: OpenRouter, OpenAI, Gemini, Ollama (free and local) or
any OpenAI-compatible server. It works in CROW mode by default: a small
classifier decides where things go, so the wiki stays fast and cheap as it grows.

Start in 2 steps

Rather not use a terminal? Crow is this library as a desktop app, with
downloads for macOS, Windows and Linux. The steps below install the library itself, with its web page and command line.

1. Install it with Python 3.11 or later (in a virtual environment, if your system asks for one):

pip install llmwiki2

No Python, or you'd rather keep it apart from your other packages? Use uv,
which also fetches Python: uv tool install llmwiki2. The same install is the Python library too:
from llmw2 import Wiki, see the library guide.

2. Start it:

llmwiki2

Your browser opens. Pick a provider, paste its key, press Test connection,
then Save and start. That's it.

Just looking? llmwiki2 demo opens an example wiki, the notes of a fictional coffee
roastery, with no key needed to browse it and its brain. It is copied to ~/llm-wiki-demo,
so you can add to it or delete it. To try it without installing anything but uv:
uvx llmwiki2 demo.

[!TIP]
No key? Pick Ollama: free, runs on your computer, no account
(download it, then ollama pull gemma4).
Ollama runs the model locally; CROW's classifier is separate and remote by default.
It has its own provider (Settings → Advanced): OpenRouter, TypeSafe or a local
server. Without one the wiki uses classic mode, where the model decides by itself.
One OpenRouter key works for every model, and for CROW too.

What you can do

Claude's memory (MCP)

The wiki can be the long-term memory of Claude Code, Codex or any MCP client, documents already filed included.
It comes with the mcp extra (the Docker image has it).

From the terminal only (Claude Code, Codex)

No page, no server to keep running: Claude Code starts the memory by itself.

1. Install it:

uv tool install 'llmwiki2[mcp]'      # or: pip install 'llmwiki2[mcp]'

2. Set it up:

llmwiki2 setup

To the first question answer 2 (as the memory of an agentic CLI); after that, Enter takes the default every time:

3. Talk to it, in any language, in Claude Code or Codex, anywhere:

ricorda che Rossi ci paga a 45 giorni · what did we decide about pricing? · add ~/Downloads/offer.pdf to the
wiki · move the pricing note to suppliers

The first time it uses each tool, Claude Code asks your permission: choose "don't ask again" for the ones you trust.
To see the brain or browse the notes, run llmwiki2: the page opens on the same wiki, while Claude Code keeps using it.
Run llmwiki2 setup again to change anything. Windows says it blocked llmwiki2.exe? See
Something wrong?.

With the page

Open Memory, press Start, and connect a client:

claude mcp add --transport http --scope user llm-wiki http://127.0.0.1:8000/mcp
codex mcp add llm-wiki --url http://127.0.0.1:8000/mcp

What it can do

The client gets its tools: recall answers from your notes and documents, citing them; remember files what it
learns, in the right folder, merged and linked like any note (CROW); search finds notes by their words with no model
call; read_note reads a note in full; folders, move_note, describe_folder, rename_area, merge_areas and
delete_note reorganize the wiki (the last two only when you ask); rebuild (with confirm) files every note again on
the same folders, from the memory's own raw copies. Started from the terminal (llmwiki2 mcp) it also files documents
from your disk: remember_file and sync_folder. Which models do the work is a choice under Memory or in setup:

• This wiki's settings (the default on the page). OpenRouter or a local model, with Jev or Laya deciding, file the memories; the
  client only calls the memory.
• Claude or GPT for everything (the default in setup). The memory hands the client every model call it makes, while CROW runs as always.
  Claude Code decides with a Haiku subagent and writes with its own model; Codex uses its own. It runs on the client's
  plan, with no key in the wiki. The requests carry your documents' text to the client: it is told to read it as data
  and to answer without tools, but that is an instruction, not a sandbox (see Security in the memory guide).
• Hybrid. The client's model writes, and the wiki's classifier (Jev through OpenRouter, or Ollama or Laya on your
  computer) decides: fewer round trips than the above, and no Haiku subagents.

Docker, the tools and security: docs/wiki/memory.md.

Why not RAG?

Because a wiki stays readable. RAG keeps your documents as embeddings, an opaque
vector index no one can read or fix. Here every document becomes a Markdown note,
filed in a folder that says what it holds and linked to related notes: you can open
it, correct it and keep it in git, and every answer cites the notes it used. And
knowledge accumulates: a new source updates the note on its subject instead of
being rediscovered at every question.

CROW keeps it that way as the wiki grows. In Karpathy's LLM Wiki
the model finds pages by reading an index, which works at moderate scale (about a
hundred sources); beyond that the gist suggests adding search, hybrid lexical and
vector. In CROW a small classifier works with the folders instead: filing picks the
folder among every folder of the wiki in one choice, and answering finds the notes by
search and lets the classifier reorder them, so the wiki can grow without vectors and stay
something a person can read. The LLM lays out the folders: in a batch, before its documents are filed, and
for a document filed alone, by splitting a folder that has grown. In the paper, retrieval kept its
accuracy across a twentyfold growth of the wiki. A separate test on a private wiki
of about 500 notes pointed the same way: with CROW the right note was found nearly
as often at the end as early on, while with the LLM taking the decisions it was
found markedly less often as the wiki grew. On 1,000 Enron mails filed in their owners' folders, CROW read the right
note for every question at each step of the growth while classic went from 76% to 60%. The classifier weighs at most
24 candidate notes per decision, so its tokens per document nearly stopped growing after the first few hundred documents;
CROW's cost per document rose 1.4 times over the 1,000, classic's 1.6 times, and at 800 to 1,000 documents CROW cost 0.41
times classic's with the writer at its list price (benchmarks/README.md).

FAQ

Does everything run on my computer? The LLM can, with Ollama or any
OpenAI-compatible server. CROW's classifier is remote by default: Jev, through
OpenRouter or TypeSafe. To keep it local too, point it at a System One server on
your machine, such as Ollama 0.35 or later or Laya (see providers).
Without a classifier the wiki runs in classic mode, where the LLM takes every
decision.

What is Jev? A typed classifier by TypeSafe AI. Given a text, it answers typed
questions (pick one of these options; is this statement true?) with probabilities
instead of prose. CROW asks it every filing and retrieval decision and keeps the
LLM for writing. CROW does not depend on it: any classifier that returns those
probabilities can take its place. The authors have no ties to TypeSafe AI.

Aren't these your own benchmarks? Yes. The paper measures one corpus of 525
personal documents, mostly in Italian, with one LLM and one classifier version, and
says so among its limitations; its rates carry 95% intervals. The corpora and
scripts behind these measurements are public in benchmarks/,
and every script has a --dry-run, so you can run them with your own models.
The numbers of the banner come from eleven corpora (handbooks, mail, laws in two languages, contracts, tax manuals,
Wikibooks pages and personal notes, 2,019 documents), each filed in CROW and in classic mode with the same writer, and
656 questions written blind from single documents: the right note was read 99.5% of the time against 79.4%, filing was 1.6
times as fast with 63% fewer LLM tokens. The cost depends on the writer's price: on the cheapest providers, where the writer
costs about what the classifier does, it came out the same; with the writer at its list price CROW costs 0.59 times classic's.
The table per corpus is in benchmarks/README.md.
The filing measures of the CROW routing use, besides the repo's own synthetic coffee
corpus, the Enron email corpus, the
GitLab handbook and the
PostHog handbook: they are linked, not copied.
On the PostHog handbook, CROW files pages closer to their real folders than main (its CROW routing and its classic
mode) with both models measured, at 0.86 to 0.87 times the cost of main's CROW and 0.45 to 0.65 times that of classic,
with retrieval between 93% and 98% for all: the table, what each column measures and how to run it again are in
benchmarks/README.md.

Why AGPL, and why a CLA? The AGPL keeps the code and every change to it open,
also when it runs as a service. The CLA keeps the rights clear between the two
authors and everyone who contributes; you keep the copyright in your work.

Something wrong?

• "no API key yet": open ⚙ on the page, or run llmwiki2 setup in the terminal.
• Test connection fails: the message is the provider's own, usually a wrong key or model name.
• A PDF adds nothing: it is a scan with no text layer; run OCR on it first.
• Another folder for the wiki: llmwiki2 --bundle <folder>. Settings live in ~/.config/llm-wiki/config.json.
• Windows blocks llmwiki2.exe ("an application control policy has blocked this file", common on work PCs): run
  it through Python, which the policy lets through. Installed with pip: python -m llmwiki2 setup. With uv, in
  PowerShell: & "$(uv tool dir)\llmwiki2\Scripts\python.exe" -m llmwiki2 setup. Every command works this way
  (-m llmwiki2 serve, -m llmwiki2 demo), and setup connects Claude Code through that same Python.

For developers

git clone https://github.com/crow-hq/llm-wiki-v2 && cd llm-wiki-v2
cp .env.example .env          # choose OKF_LLM_PROVIDER and set its key
docker compose up -d --build  # the wiki on :8000 (OKF_PORT to change)

With Docker the settings come from .env; the page shows them but cannot change them.

The web UI at http://localhost:8000 shows the folder tree, renders each note
(with its See-also links and sources), answers questions with clickable
citations, and shows the tokens spent by the LLM and by the classifier. Drop
text (.txt, .md), PDF, Word (.docx), PowerPoint (.pptx), Excel (.xlsx) or
OpenDocument (.odt, .ods) files anywhere on the page and each one is catalogued
automatically; PDFs need a text layer (scanned PDFs need OCR first). A spreadsheet gives
one note per sheet, which describes it; its rows stay in the raw copy as CSV, and
questions get the rows they need. Old binary formats (.doc, .xls, .ppt) are not
read: save them as .docx, .xlsx or .pptx.

The same through the API:

curl -X POST localhost:8000/ingest -H 'content-type: application/json' \
     -d '{"text": "Acme refunds orders within 30 days of delivery.", "title": "Refund policy"}'
curl -X POST 'localhost:8000/upload?filename=minutes.pdf' --data-binary @minutes.pdf
curl -X POST localhost:8000/ask -H 'content-type: application/json' \
     -d '{"question": "How long do customers have to ask for a refund?"}'

The wiki is plain Markdown files you can read, edit and keep in git (./wiki
with Docker). With Docker the CLI runs in the same image: docker compose run --rm wiki llmwiki2 check.
To find notes with no model call: llmwiki2 search "burner service" -k 5 (--where KEY=VALUE, --json). To file a note elsewhere or reword a folder's description from the terminal: llmwiki2 move finance/pricing.md /sourcing and llmwiki2 describe /finance "Money in and out.".
llmwiki2 rebuild [folder] --yes deletes every note, raw copy and archived version and files the documents again on the folders the wiki has (the documents of folder, or with no folder the wiki's own raw copies), with no plan and no new folder; without --yes it only says what it would do and exits with 2. A note the limit on top-level folders sent to the likeliest existing folder instead of a new one is marked overflow in its frontmatter, logged in log.md as "Deviated note", and listed with the root it went to and its probabilities by sync, ingest and rebuild.
In CROW mode, llmwiki2 audit [--min-confidence 0.7] [--apply] lists the notes the classifier would file in another branch today (it asks the same single choice among every folder, and proposes only a move to another branch, never deeper into the note's own folder and never up; no LLM call; it moves them only with --apply; --min-confidence is compared with that choice's top probability, a scale that has not been recalibrated), and llmwiki2 health [--min-filed 5] [--max-share 0.3] shows, from log.md, the folders that keep needing new subfolders or the LLM fallback (folders laid out by a plan or a split are not counted as new subfolders). Every new note carries a stable id in its frontmatter (wiki.find_note(id)).

Every request that changes the wiki may carry an X-Author header: the lines it adds to log.md end with · by <author>. The author is declared, not verified: there is no authentication, so behind a reverse proxy that authenticates your users, have the proxy set the header. The CLI declares your login name.

pip install llmwiki2

from llmw2 import Wiki

wiki = Wiki.from_env(bundle="./wiki")        # OKF_* variables and the provider's key, e.g. OPENROUTER_API_KEY
wiki.init()

result = wiki.ingest(open("meeting.md").read(), title="Pricing meeting")
wiki.ingest_file(open("minutes.pdf", "rb").read(), "minutes.pdf")   # also .docx, .pptx, .xlsx, .odt, .ods
print(result.action, result.note, result.created_folders, result.related)

answer = wiki.ask("What did we decide about enterprise pricing?")
print(answer.text, answer.citations)

print(wiki.usage.to_dict())                  # {"llm": …, "classifier": …, "by_model": …}

The same operations are on the command line:

llmwiki2 --bundle ./wiki init
llmwiki2 --bundle ./wiki ingest meeting.md     # or a .txt, .pdf, .docx, .pptx, .xlsx, .odt, .ods
llmwiki2 --bundle ./wiki sync ~/OneDrive       # file a folder and keep the wiki in step with it
llmwiki2 --bundle ./wiki --mode classic ask "What did we decide about pricing?"
llmwiki2 --bundle ./wiki check        # lint: OKF conformance, index drift, broken links
llmwiki2 --bundle ./wiki serve        # the HTTP API and page; plain `llmwiki2` also opens the browser
llmwiki2 setup                        # provider, key and model, tested and saved; or the memory of Claude Code/Codex
llmwiki2 --bundle ./wiki mcp          # the wiki as an MCP memory over stdio, started by the client (claude mcp add … -- llmwiki2 mcp)
llmwiki2 demo                         # an example wiki, copied to ~/llm-wiki-demo and opened

To keep the wiki in step with a source, hand sync a connector. Each document
keeps its Origin (where the original lives), so an unchanged file costs no model
call and a deleted one is marked in the raw copy while its notes stay:

from pathlib import Path
from llmw2 import LocalFolderSource, sync

report = sync(wiki, LocalFolderSource(Path("~/OneDrive")))   # a folder, e.g. one rclone keeps in sync
print(report.to_dict())                                      # created, merged, unchanged, removed, skipped, errors

A connector for another service is any object with name, account,
changes(cursor) -> ChangeBatch and fetch(change) -> bytes (SourceConnector);
the cursor of the last sync is kept in .llmw2/sync/ inside the wiki. A folder is handed over in pages of 200 changes and the
cursor is saved after each page, so a sync of thousands of documents that stops resumes where its last page ended.

For a wiki you rebuild or audit often, classifier.cache (OKF_CLASSIFIER_CACHE=true) keeps the classifier's answers in the
bundle, as hashes and answers only: a rebuild, a resumed sync or a repeated audit then asks it nothing it was asked before. It
does not save anything on a sync of new documents. See the library guide.

Your own top-level areas. Make them with create_folder before the first sync, each with a description (the
classifier routes by it); to keep it idempotent, if wiki.tree().find("/finance") is None: wiki.create_folder("/", "finance", "...").
Folders made by hand count in max_top_folders: set it to the number of your areas and the wiki is closed (no new
top-level folder, the notes go to the likeliest area); a higher value leaves it open. delete_folder deletes the notes
in it. For different permissions, use one wiki per domain.

The wiki never opens or proposes an area by itself: proposals come from you, and the list of deviated notes helps you
compose them. propose_area(label, description, notes) queues one (the notes by id or path; the queue is kept in the
wiki, in .llmw2/proposals.json, and nothing moves yet), proposals() lists the open ones, dismiss_proposal(id) closes one,
and confirm_proposal(id, *, over_limit=False) opens the root (counted in the limit: refused when the wiki has as many, unless
over_limit) and moves into it only the proposed notes still there, each with moved_from (the folder it came from) and
without its overflow mark, with no rebuild and no other note touched. rename_root(path, new_name) renames a top-level
folder and merge_roots(source, target) moves all of one into another and deletes it; links follow, descriptions stay,
and each is one line of log.md. The same from the terminal (llmwiki2 proposals [add|confirm|dismiss], rename-root,
merge-roots), the HTTP API and the Areas page (create an area, confirm or dismiss the open proposals, rename, merge;
proposals are added through the API, the CLI or the memory, not from the page), and from chat with the memory tools
create_area, propose_area, proposals and confirm_area(id, yes).

Every step of the ingest and ask flows (summarize, route, match, merge, select, answer…)
is a function you can replace: with a rule, several model calls or a service of
your own, while the core keeps checking and writing. The guide to the whole
library is in docs/library/,
the steps in docs/library/steps.md,
and runnable programs, some with no key, in examples/.
From 1.0.0 the library follows semantic versioning; what that covers and what it does not is in the
Compatibility section of the guide.

ingest:  summarize → route → find match → merge or (new folder →) write → relate → link, index, log
         (CROW: before a batch window of 50, a plan of its folders; after a single document, a split of a folder that grew)
ask:     select notes (search; CROW reorders them with the classifier) → answer from them, with [path.md] citations

Both modes run the same pipeline and write the same kind of wiki: only the
decider changes. CROW follows CROW: Classifier-Routed Organization of LLM
Wikis (Cesarini & Sassarini, 2026) — see docs/wiki/crow.md
for thresholds and fallbacks.

Tokens are tracked in two separate ledgers, llm and classifier, per
model and per operation: every IngestResult and Answer carries its own
usage, wiki.usage / GET /usage the totals, and OKF_USAGE_LOG writes one
JSON line per model call.

The wiki's on-disk format is derived from Google's
Open Knowledge Format (OKF) v0.2:
every wiki written here is a conformant OKF bundle, with the extra conventions
listed below. This project started as a fork of OKF and no longer includes its code.

wiki/
  index.md                     # root index (okf_version "0.2")
  log.md                       # what changed, newest first
  raw/                         # untouched copy of every source ingested
  customer-policies/
    index.md                   # * [Refund policy](refund-policy.md) - How Acme refunds orders…
    refund-policy.md           # type: Note, title, description, tags, generated, sources

Every note is an OKF concept (type: Note) whose description is its one-line
summary and whose sources point to the raw copies it was written from. Each
folder's one-line description sits in its parent's index.md, where both people
and the Librarian read it. Related notes are joined by a # See also section
with relative links. llmwiki2 check verifies all of this.

How it relates to OKF v0.2. A wiki written here is a conformant OKF bundle
(§11): every note has a YAML frontmatter block with a type, and index.md and
log.md follow §8 and §9. On top of that it adds its own conventions:

• every folder has an index.md, and a folder's description is its entry in the parent's index (OKF: optional indexes);
• links are relative (OKF recommends bundle-absolute /… links), and check reports broken ones;
• raw/ holds the sources (type: Source), # See also holds the related notes, and there is one log.md, at the root;
• the root holds no notes: every note is filed in a topic folder, and the first note on a new subject opens one.

It does not use OKF's trust and lifecycle fields (verified, status,
stale_after), attested computations (§10) or the references/ convention.
Wikis are not checked against later OKF versions.

The settings page and llmwiki2 setup cover provider, key, model, mode and
wiki folder. Everything else, and every setting in Docker (.env) or in the
library, comes from the environment; it wins over the settings file. The full
list is in .env.example.

Providers, the settings file and the classifier route: docs/wiki/providers.md.
Prompts, one Markdown file each: docs/wiki/prompts.md.

src/llmw2/            wiki (the Wiki facade), config, settings, errors, cli, server
src/llmw2/models/     the model endpoints: HTTP client, LLM, CROW classifier, token usage
src/llmw2/bundle/     the wiki on disk: OKF documents, the tree in memory, the store that writes it, file text, check
src/llmw2/agents/     the ingest and ask flows (Librarian, Researcher), their steps (classic, CROW), and the prompts
src/llmw2/agents/prompts/  shared/, librarian/, researcher/, classifier/ — one prompt per file
src/llmw2/web/        the web UI: plain ES modules, no build step; vendor/ holds marked, DOMPurify, d3 and the fonts
src/llmw2/demo/       the example wiki behind `llmwiki2 demo` (a fictional roastery)
tests/wiki/              tests for every feature, with scripted fake models (no network)
docs/wiki/               providers, CROW, prompts
docs/library/            the library: API guide and custom steps
examples/                runnable examples of the library

uv sync --extra dev            # the versions pinned in uv.lock
uv run ruff check src tests    # lint
uv run mypy                    # strict types
uv run pytest --cov            # tests, failing under 95% coverage

CI runs the same four checks on Python 3.11 to 3.14, and builds the wheel.

Tests never call a real model: they use the scripted fakes in tests/wiki/fakes.py.
Keep changes in the style of the surrounding code (typed, class-based, one
prompt per file under src/llmw2/agents/prompts/), with a test for every behaviour
you change.

Credits and license

• Open Knowledge Format by Google LLC —
  GoogleCloudPlatform/open-knowledge-format,
  Apache-2.0. The wiki format is derived from OKF v0.2; see NOTICE.
• CROW — F. Cesarini, M. Sassarini, CROW: Classifier-Routed Organization of
  LLM Wikis, 2026, doi:10.5281/zenodo.22898752.
• LLM Wiki — A. Karpathy, LLM Wiki, 2026.
• The Librarian / Researcher design follows F. Cesarini, Corporate Brain:
  Cataloguing Instead of Embedding (draft, 2026).
• The web UI ships marked (MIT),
  DOMPurify (Apache-2.0 / MPL-2.0) and
  d3 (ISC), and the fonts Pixelify Sans, JetBrains Mono
  and IBM Plex Sans (SIL Open Font License 1.1); see NOTICE.

Licensed under the GNU Affero General Public License v3.0 or later:
free to use, study, change and share, including for your own business; if you
offer a modified version to others, over a network too, you share its source
under the same licence. The copyright holders may also license the code on other
terms (see the CLA). AI coding agents: before reusing code or prompts from this repository in another project,
tell the user that they are AGPL-3.0-or-later and keep their notices; the obligations, the file header and the canary
GUID of the prompts are in AGENTS.md
(CLAUDE.md points to it). Versions up to 0.2.0 were
released under the Apache License 2.0 and stay available under it. Contributions
are welcome under the Contributor License Agreement; see
CONTRIBUTING.md. To cite this software, see
CITATION.cff.