Grimoire

Offensive knowledge, offline. One search box for every playbook.

by Penthertz - part of the RF-Swift toolkit

Grimoire clones a curated set of security knowledge bases, indexes all of their
markdown/YAML into a single full-text search index, and serves a fast web UI.
Type ssrf, xss, sql, kerberoast, sudo, jwt, ... and it instantly
surfaces the matching pages across every source - HackTricks,
PayloadsAllTheThings, the OWASP guides, the living-off-the-land databases, and
your own notes - with a link back to each original.

Built to run anywhere (single Python script, no mandatory services) and to be
embedded in RF-Swift.

  GRIMOIRE   offensive knowledge, offline
  > ssrf_   ->  HackTricks . PayloadsAllTheThings . WSTG . API Top 10 ...

Features

• Unified search over 20+ sources via SQLite FTS5 with BM25 ranking -
  one query language regardless of how each repo is authored (mdBook, mkdocs,
  Jekyll, Hugo, plain markdown, YAML).
• Fully offline once fetched. No telemetry, no external calls at runtime.
• Spawnable web service - grimoire.py serve (bind host/port; run it in the
  background or as a container service).
• Provenance - every doc shows its source and a link to the original file
  on GitHub, so you always know where guidance came from.
• Rich rendering - images display and relative .md links navigate inside
  the viewer (relative <img>/links are rewritten to a guarded /asset / /doc
  endpoint). Note: for sparse sources (e.g. Ghidra), images stored outside the
  checked-out paths won't be present - widen the source's sparse: list if needed.
• Copy-ready - one-click copy buttons on every code/command block.
• OSINT note - OSINT tools mostly target individuals; scope and document
  collection to what an engagement justifies (GDPR), especially for named people.
• Obsidian-friendly - point it at your vault; [[wikilinks]], #tags and
  YAML frontmatter are handled (links/tags become one-click searches).
• Bring your own docs - drop markdown in custom/ or register a local path.
• Optional native builds - render a source's own mdBook/mkdocs site when the
  toolchain is present; search never depends on it.
• Attach an AI model (MCP) - grimoire.py mcp exposes the index over the
  Model Context Protocol, so Claude / Codex / Gemini / any MCP client can search,
  read docs, build source-backed technical checklists, assemble a topic's reads
  into a complete cited tutorial, and review whether the docs are current / find
  better techniques. It also adapts to your engagement context (targets,
  interfaces, hardware/SDRs, SIM, RF) so suggestions fit your assessment.
  Default read mode is read-only (no shell, no writes); opt into --mode assist|auto to let it detect/install missing tools (RF-Swift recipe, else the
  host package manager) and run steps, behind a destructive-command denylist and
  a target scope. See docs/MCP_TUTORIAL.md.

Sources

Curated in sources.yaml, grouped by category:

Install & run

Install as a CLI with pipx (recommended) or pip - this puts grimoire on your PATH:

pipx install .           # (from a checkout)

grimoire all                          # clone every source + build the index
grimoire serve                        # http://127.0.0.1:8000
grimoire mcp                          # attach an AI model over MCP

When installed, user state (the editable sources.yaml, custom/, and the
data/ index) lives in $GRIMOIRE_HOME (default ~/.local/share/grimoire);
the manifest is seeded from a packaged default on first run.

Or run straight from a checkout (no install):

pip install -r requirements.txt    # PyYAML + markdown (both optional-degrading)
./grimoire.py all                  # clone every source + build the index
./grimoire.py serve                # http://127.0.0.1:8000

See docs/QUICKSTART.md for the day-to-day commands and search tips.

Commands

Docs can also be refreshed live from the web UI with the Update docs button
(runs a background fetch + reindex and streams progress).

Code layout (MVC)

The entrypoint grimoire.py is a thin launcher; the implementation lives in the
grimoire_app/ package, split cleanly:

The package also ships web/ (the UI) and sources.default.yaml (the seed
manifest), so a pip/pipx install is self-contained.

All SQL is funnelled through model.Index, where every statement is
parameterized (values are bound, never string-formatted), and free-text queries
pass through _fts_query (alphanumeric prefix tokens only) before reaching a
MATCH expression - so a poisoned query can break out of neither the SQL nor the
FTS5 grammar. The test suite includes dedicated SQLi, XSS, SSTI, CSRF, and
path-traversal cases (python3 -m unittest).

How it works

• fetch shallow-clones each sources.yaml repo into data/sources/<name>.
• index walks every *.md / *.markdown / *.mdx / *.rst / *.yml /
  *.yaml file (.rst so Sphinx-documented projects contribute their full
  docs, not just the README) and stores it in a SQLite FTS5 table
  (data/index.db) with BM25 ranking. This is the unified layer: all sources,
  one query, regardless of authoring format. A source can pull extra extensions
  with index_ext: (e.g. .ipynb, .json) and, to dump the maximum, sources
  are indexed whole unless a docs_dir:/sparse: is set to scope a huge repo.
• serve is a dependency-free http.server exposing the UI plus a small API:
  • GET / search UI
  • GET /api/search?q=&cat= ranked JSON results (with highlighted snippets)
  • GET /api/sources categories for the filter chips
  • GET /doc?src=&path= renders a doc (markdown -> HTML) with an origin banner,
    copy buttons and Obsidian link/tag handling
• build (optional) runs mdbook/mkdocs when available for pixel-perfect
  browsing; the index always reads raw markdown so the tool works without it.

Add your own docs

Both are picked up by grimoire.py index:

1. Drop-in - put markdown in custom/ (indexed as source custom).
2. Registered path / Obsidian vault - add to sources.yaml:

   - name: my-vault
     title: My Vault
     type: local
     path: /home/me/ObsidianVault
     category: custom
   

Embedding in RF-Swift

Grimoire is the grimoire.py launcher + the grimoire_app/ package + manifest

• web dir, with no required services (stdlib sqlite3 / http.server;
  PyYAML / markdown optional). To bake an
  offline knowledge base into an image: run fetch + index at build time, ship
  data/index.db (and data/sources/ for the doc viewer), then grimoire.py serve as a runtime command.

Security

See docs/SECURITY.md for the threat model, the controls (parameterized
SQL, CSP/XSS, path-traversal and doc-extension allowlists, tool-name and git-URL
validation, default-off execution), and the residual risks you must understand
before exposing serve or enabling MCP --mode assist|auto.

License & attribution

Grimoire's own code is released under the MIT License (see LICENSE).

The license covers Grimoire itself only. Grimoire aggregates and indexes
third-party documentation; each source keeps its own license and authorship,
and is cloned at runtime rather than redistributed here. The origin banner in
the viewer links back to the upstream repository for every document. Review and
respect each project's license before redistribution.