Tesseract

mcp
Security Audit
Warn
Health Warn
  • License — License: MIT
  • Description — Repository has a description
  • Active repo — Last push 0 days ago
  • Low visibility — Only 5 GitHub stars
Code Pass
  • Code scan — Scanned 12 files during light audit, no dangerous patterns found
Permissions Pass
  • Permissions — No dangerous permissions requested

No AI report is available for this listing yet.

SUMMARY

Persistent, shared memory for AI agents: an MCP server over a plain-markdown Obsidian vault with hybrid search, a browsable knowledge graph and a self-maintaining caretaker

README.md
tesseract-mcp: a modular MCP server for multi-agent systems

Persistent, shared memory for AI agents, living in a plain-markdown Obsidian vault.

CI
Cross-platform
License: MIT
Python 3.11+
MCP server
Obsidian
25 tools
Retrieval MRR 0.87

About · Architecture · Features · Quickstart ·
Client configuration · Tool reference ·
Configuration · Operator CLIs ·
Retrieval quality · Contributing ·
Provenance and reuse · License

About

Agents forget. Every new session of Claude Code, Claude Desktop or Codex starts
from zero, and whatever one agent learned on one machine is gone for the next.
Tesseract fixes that with the most boring storage there is: a folder of
markdown files, open in Obsidian, synced between machines with LiveSync.

tesseract-mcp is the MCP server in front of that folder. Any MCP client can
connect to it and get the same 25 tools: search the vault, read and write
notes, walk the knowledge graph, log a session, file a task. Several agents
(a planner, an engineer, a researcher, a scheduled caretaker) read from and
write to one memory, and the human owns it, in an editor they already use.

Agents connect through tesseract-mcp to the vault's code, data, web and file resources

What makes it different from "a vector database with an MCP wrapper":

  • Hybrid retrieval. BM25 and embedding similarity combined as a weighted sum of normalised scores.
  • A knowledge graph you can open. Entities extracted by an LLM are written as real markdown notes under Claude/Graph/, so the graph is browsable in Obsidian's graph view, not locked in a database.
  • A write contract enforced in code. Agents write freely under Claude/; everything else needs explicit confirmation.
  • Self-maintaining. One scheduled Librarian sweep indexes, files and reports what needs repair.
  • Rebuildable state. Every cache lives under ~/.tesseract-mcp/ and can be deleted and rebuilt from the vault.

Who it's for

  • You run more than one AI coding agent and want them to share what they learn.
  • You already keep notes in Obsidian and want agents to read and extend them without trashing the vault.
  • You want memory that survives a machine wipe: delete every cache, and the markdown rebuilds it.

Status

Used daily on a private vault; the public repo is the server, the caretakers
and a synthetic eval corpus. Windows, macOS and Linux are all supported and
tested in CI. Treat the organizer and Librarian as power tools: run them
with --dry-run first (the CLI makes you).

Architecture

The system is drawn on five trust bands. Arrows point from the caller to what
it depends on.

flowchart TB
    subgraph OUT["Outside world"]
        direction LR
        CL["MCP clients<br/>Claude Code · Claude Desktop · Codex"]
        ING["Ingest servers<br/>fetch · arxiv<br/>(untrusted content)"]
        SYNC["Other machines<br/>Obsidian + LiveSync"]
    end

    subgraph MODEL["Model boundary"]
        direction LR
        EXT["Extractor CLI<br/>claude -p · codex exec"]
        EMB["Embeddings<br/>Smart Connections vectors<br/>bge-micro-v2 fallback"]
    end

    subgraph CORE["Deterministic core · tesseract-mcp"]
        direction LR
        SRV["FastMCP server<br/>25 tools"]
        HYB["Hybrid search<br/>BM25 + cosine → weighted sum"]
        GRAPH["Graph store<br/>+ SQLite cache"]
        ORG["Organizer<br/>neighbor-vote filing"]
        LIB["Librarian<br/>scheduled sweep"]
        VLT["vault.py<br/>path containment +<br/>Claude/ write quarantine"]
    end

    subgraph TRUTH["Ground truth"]
        direction LR
        MD[("Obsidian vault<br/>markdown notes")]
        GN[("Claude/Graph/<br/>entity notes")]
        DB[("CouchDB<br/>LiveSync replica")]
    end

    subgraph HUMAN["Human interlocks"]
        direction LR
        DRY["Dry-run defaults<br/>organize · consolidate"]
        CONF["confirm_outside_claude"]
        SCH["_schema.md<br/>blesses a sheet folder"]
        UNDO["undo_move · retirement<br/>proposals · --dry-run"]
    end

    CL -->|MCP stdio| SRV
    ING -.->|clipped into notes| SRV
    SRV --> HYB & GRAPH & ORG & VLT
    LIB --> GRAPH & ORG
    HYB --> EMB
    GRAPH --> EXT
    ORG --> EMB
    VLT --> MD
    GRAPH --> GN
    MD <-->|LiveSync| DB
    DB <-->|LiveSync| SYNC
    VLT -.-> CONF & SCH
    ORG -.-> DRY & UNDO
    GRAPH -.-> DRY

Markdown is authoritative. The search index is held in memory and brought up to date
from the files before each search, and the
embedding cache and graph.db mirror live in a per-vault state directory, so
only the vault itself ever travels through LiveSync. Full detail:
docs/ARCHITECTURE.md.

Retrieval pipeline

flowchart LR
    Q[query] --> F[candidate filter<br/>tags / folder]
    F --> B[BM25L keyword score<br/>text plus file name]
    F --> E[cosine similarity<br/>Smart Connections vectors,<br/>bge-micro-v2 fallback]
    B --> S[min-max normalise,<br/>weighted sum 0.5 / 0.5]
    E --> S
    F -.->|only when no note has a keyword score| SUB[substring match]
    SUB -.-> S
    S --> W[entity notes x 0.5,<br/>note named by the query first]
    W --> H[ranked hits with excerpts]

Every candidate is scored; each lane is scaled to 0-1 over the candidates and
the two are added. Substring matching only stands in for the keyword score for
queries the tokenizer can't match, such as single characters or punctuation.

The Librarian sweep

flowchart LR
    I[index<br/>drain changed notes] --> O[organize<br/>file new notes]
    O --> C[cache<br/>rebuild graph.db if needed]
    C --> CL[cleanup<br/>report repairs, prune caches]
    CL --> CO[consolidate<br/>throttled, proposals only]
    CO --> H[health checks]
    H --> RP[report<br/>Claude/Librarian.md]

Each step is isolated: a failure records an error and later steps still run.
The CLI exits non-zero when any step failed, so a scheduler can alert on it.

Features

Modular, multi-agent, tool access, Tesseract core
Area What it does
Hybrid search BM25L and embedding cosine similarity over the same candidate set, normalised and added as a weighted sum. Entity notes are weighted down and a note named by the query goes first. Reuses Obsidian Smart Connections vectors when fresh; falls back to the same model (bge-micro-v2) locally so the vector space is never mixed.
Semantic graph An LLM pass extracts people, organizations, domains, topics, projects and sources into entity notes under Claude/Graph/, mirrored into SQLite for traversal. related_notes ranks connections by strength (each shared entity adds 1/hop, so direct and multiple links win), caps the list (default 20), and explains each one with its entity chain.
Write contract vault.py checks every path agents write through and quarantines those writes to Claude/. Writing elsewhere requires confirm_outside_claude=True. The one caretaker exception is the organizer's moves and link rewrites in the human tree, under the constitution's standing permission; they never edit sheet rows. The human-readable rules ship as a constitution note.
Structured sheets A human folder becomes agent-writable only when it holds a _schema.md. sheet_upsert validates every field against that schema.
Organizer Files new notes by embedding neighbor vote. A share of ≥ 0.7 moves the note; below that, an unfiled note becomes a proposal for a human and an already-filed note stays put. Every move is journaled and reversible with undo_move.
Graph cleanup Finds mentions of deleted notes, dangling relations and merge-stub chains, and prunes derived caches. The sweep reports the repairs; cleanup --apply-repairs applies them (or opt in with --auto-cleanup). Retraction holds when so many notes are missing that the vault looks only partly present. The Librarian only proposes entity retirements; cleanup --apply-retirements applies them.
Consolidation Finds duplicate entities with an LLM pass. Dry-run by default; the Librarian only ever proposes merges.
Recall harness Claude Code skills that turn the vault into memory you can question. See Skills.
Provisioning One command installs a pinned set of 14 Obsidian plugins, seeds settings, and installs the Claude/ conventions tree. Every plugin asset is checked against a reviewed SHA-256 lock before anything is written (bumping a pin). --check reports drift.
Session hooks A SessionStart hook injects project recall into new Claude Code sessions; a Stop hook nudges log_session after significant work. Both fail silent.

Skills

Installed with python -m tesseract_mcp.skill_sync from skills/.

Skill Purpose
/recall Researched answer from the vault, every claim cited as a [[wikilink]]; filed into Claude/Answers/
/resume Briefing on where a project left off
/digest Review of what changed: notes, captures, tasks, health
/connections Graph serendipity: what links to a topic
/clip Compile one external source into a cited Inbox note
/distill Promote repeated captures into evergreen concepts
/decide Record a decision in the append-only ledger
/rounds Read-only health inspection of the caretakers
/dossier Deep multi-section briefing on a topic

Quickstart

Setting this up through an AI agent? Point it at docs/AGENT_INSTALL.md.
It asks for your vault path first, shows you each change before it writes, and leaves client config edits to you.

Requirements: Python 3.11+, an Obsidian vault, and the claude or codex
CLI on PATH for entity extraction. Windows commands come first; the
macOS / Linux steps follow.

git clone https://github.com/DeepExtrema/tesseract-mcp
cd tesseract-mcp
python -m venv .venv
.venv\Scripts\pip install -e .

Provision a vault (plugins, settings, conventions tree). It is idempotent:
settings and conventions files are only written where none exist, and a
plugin is only (re)installed when its version differs from the pin.

.venv\Scripts\python -m tesseract_mcp.provision <path-to-vault>

Register the curated MCP server set (tesseract plus fetch and arxiv ingest)
into Claude Code. Sync is additive: existing entries are never changed.

$env:TESSERACT_VAULT_PATH = "<path-to-vault>"
.venv\Scripts\python -m tesseract_mcp.mcp_sync --check   # preview
.venv\Scripts\python -m tesseract_mcp.mcp_sync           # apply

Then open the vault once in Obsidian (disable Restricted Mode, finish LiveSync
setup) and call the index_brain tool to build the graph.

macOS / Linux

Same flow with the POSIX venv layout (.venv/bin/):

git clone https://github.com/DeepExtrema/tesseract-mcp
cd tesseract-mcp
python3 -m venv .venv
.venv/bin/pip install -e .
.venv/bin/python -m tesseract_mcp.provision <path-to-vault>
export TESSERACT_VAULT_PATH="<path-to-vault>"
.venv/bin/python -m tesseract_mcp.mcp_sync --check   # preview
.venv/bin/python -m tesseract_mcp.mcp_sync           # apply

mcp_sync and hook_sync pick the right venv binary on every OS. Set
TESSERACT_VAULT_PATH wherever the hooks run too: their built-in default
is the Windows path C:\Vaults\Tesseract. If the vault is already synced
another way (iCloud Drive, for example), skip LiveSync rather than running two
syncs on one vault.

MCP client configuration

The server is a stdio process started by the tesseract-mcp console script.
It needs one environment variable, TESSERACT_VAULT_PATH.

Claude Code: mcp_sync does this for you; the manual equivalent is:

claude mcp add --scope user tesseract -e TESSERACT_VAULT_PATH=C:\Vaults\Tesseract -- C:\path\to\tesseract-mcp\.venv\Scripts\tesseract-mcp.exe

Claude Desktop: add to claude_desktop_config.json:

{
  "mcpServers": {
    "tesseract": {
      "command": "C:\\path\\to\\tesseract-mcp\\.venv\\Scripts\\tesseract-mcp.exe",
      "env": { "TESSERACT_VAULT_PATH": "C:\\Vaults\\Tesseract" }
    }
  }
}

On macOS and Linux the command is <repo>/.venv/bin/tesseract-mcp. Any MCP
client that can launch a stdio server works the same way. In a new
session, call onboard first.

Tool reference

25 tools, grouped by what they touch. Write tools stay inside Claude/
unless a call explicitly confirms otherwise.

Group Tool Purpose
Orient onboard Call first: constitution, routing, cheat-sheet, graph status
Retrieve search_brain Hybrid search with optional tag/folder filters
context_bundle Search hits plus their graph entities and related notes in one call; the related list is capped at 20
recall_bundle Read-only raw material for /digest (mode="digest") and /resume (mode="resume")
read_note Read any note
query_notes Filter notes by frontmatter metadata
get_backlinks Notes whose [[wikilinks]] point at a note
list_recent Recently modified notes
list_tasks Checkbox tasks across the vault
Write log_session Session note in Claude/Sessions/
capture Timestamped thought in today's Claude/Inbox/ note
upsert_concept Create or extend an evergreen note in Claude/Concepts/
write_note General write; outside Claude/ only with confirm_outside_claude=True
add_task Obsidian Tasks checkbox in Claude/Tasks.md
Sheets sheet_upsert Schema-validated create/update of one row in a registered sheet
sheet_query Typed filter/sort over a sheet's rows
sheet_schema List sheets, or show one sheet's full contract
Graph index_brain Extract entities from new/changed notes; call again while remaining > 0
find_entity Look up entities by name or alias, optional type filter
related_notes Notes connected through shared entities, ranked by strength and capped by limit (default 20), with the chain
graph_stats Entity, edge and mention counts
consolidate_graph Merge duplicate entities (dry-run by default)
Organize organize_vault Neighbor-vote filing sweep (dry-run by default)
librarian_status Last sweep's results, health checks and pending proposals (read-only)
undo_move Reverse the organizer's most recent move of a note

Configuration

Variable Default Purpose
TESSERACT_VAULT_PATH required Vault root the server opens
TESSERACT_STATE_DIR ~/.tesseract-mcp/<vault-hash>/ Where caches and Librarian state live
TESSERACT_EXTRACTOR codex Extraction backend: codex or claude
TESSERACT_EXTRACT_MODEL haiku Per-note extraction model (claude backend only)
TESSERACT_CONSOLIDATE_MODEL sonnet Consolidation model (claude backend only)
CLAUDE_SETTINGS_PATH ~/.claude/settings.json Settings file hook_sync writes hooks into
TESSERACT_OWNER the vault owner Owner name written into new vault-root AGENTS.md and CLAUDE.md; existing guides are never rewritten

mcp-servers.json is the curated server manifest and
vault-template/plugins.json pins the Obsidian plugin versions.

Operator CLIs

These run outside the MCP server and are never exposed as tools.

Command What it does
python -m tesseract_mcp.librarian <vault> [--dry-run] [--max-moves N|unlimited] [--max-moves-per-day N|unlimited] The scheduled caretaker sweep. Default changed: organizer moves are now capped at 10 per run and 20 per day (previously unlimited)
python -m tesseract_mcp.indexer <vault> [--rebuild-only] Index directly, or rebuild graph.db from the markdown
python -m tesseract_mcp.organize <vault> [--dry-run] Run only the filing sweep
python -m tesseract_mcp.cleanup <vault> [--apply-repairs [--force]] [--apply-retirements] Report deleted notes and orphaned entities; apply the mechanical repairs or retire the orphans on request
python -m tesseract_mcp.consolidate <vault> [--apply] Propose duplicate-entity merges; apply them on request
python -m tesseract_mcp.sheets <vault> --check Validate every sheet row against its schema
python -m tesseract_mcp.provision <vault> [--check] Provision or check a vault
python -m tesseract_mcp.mcp_sync [--check] Register the MCP server set in Claude Code
python -m tesseract_mcp.skill_sync [--check] Install the skills into ~/.claude/skills
python -m tesseract_mcp.hook_sync [--check | --install] Install the SessionStart and Stop hooks
python -m tesseract_mcp.evals Run the retrieval benchmark

Scheduled maintenance

The shipped Librarian schedules (scripts/librarian-task.cmd and the Librarian launchd plist template) pass no move flags, so they get the new defaults: at most 10 organizer moves per run and 20 per day. Held-back candidates are counted in the run report and Claude/Librarian.md and re-evaluated next sweep; edit the schedule to add --max-moves / --max-moves-per-day (N or unlimited) to change that.

First Librarian run on a real vault: use --dry-run and review the
report before scheduling unattended runs. Dry-run writes no state, moves,
report, throttle baselines or vector caches. It still calls the
consolidation model so the report can show merge proposals. The CLI
enforces this: on a vault with no prior sweep, an applying run refuses
until you pass --confirm-first-run once by hand. scripts/librarian-task.cmd is the Windows
Task Scheduler entry point.

On macOS, schedule with launchd instead. Each template says how to install
and remove it. Install one of the two, not both (if both fire, a caretaker
lock in the state dir makes the second one refuse rather than write at the
same time):

Template Runs
scripts/com.tesseract.librarian.plist.template The full Librarian sweep daily at 04:00 (includes indexing)
scripts/com.tesseract.index.plist.template scripts/index-nightly.sh at 04:30: incremental indexing only, no file moves

macOS privacy: a launchd job can't read a vault in iCloud Drive
(~/Library/Mobile Documents/) or another protected folder unless the
program launchd starts (/bin/bash, or the venv python) has Full Disk
Access in System Settings > Privacy & Security. Without it,
index-nightly.sh exits with cannot read vault. Grant it deliberately,
or keep the vault outside protected folders.

Trust boundaries

  • The Claude/ write quarantine is enforced by the MCP server's write tools.
    The operator CLIs above also write elsewhere (.obsidian/plugins,
    vault-root AGENTS.md and CLAUDE.md, organizer moves), so review them
    before pointing them at a vault you care about.
  • Content arriving through fetch and arxiv is untrusted. It lands in
    notes, and the extractor reads full note text, so treat anything recalled
    from ingested sources as data, not instructions.

Retrieval quality

Latest recorded baseline (2026-11-15), from 17 golden queries run against the
production hybrid_search path with real bge-micro-v2 embeddings over the
synthetic fixture corpus in evals/:

success@1 success@5 success@10 recall@5 recall@10 MRR
0.82 1.00 1.00 1.00 1.00 0.87

The corpus is synthetic because this repo is public; a private golden set
lives in the vault. Reproduce with python -m tesseract_mcp.evals.

Repository layout

src/tesseract_mcp/   server, retrieval, graph, organizer, librarian, CLIs
skills/              Claude Code skills (/recall, /resume, /digest, ...)
scripts/hooks/       SessionStart and Stop hooks
vault/               constitution and root guide installed into vaults
vault-template/      pinned Obsidian plugins and settings
evals/               synthetic fixture vault and golden queries
server/              CouchDB + Caddy stack for LiveSync
docs/                architecture, design specs and plans
tests/               pytest suite

Contributing

.venv\Scripts\pip install -e ".[dev]"
.venv\Scripts\python -m pytest -q
  • Base branch is main. Open PRs against it; CI runs the full suite on every PR (ci.yml).
  • Keep markdown authoritative: anything the server derives must be rebuildable from the vault.
  • Never import numpy, torch or sentence-transformers lazily inside a tool body. Warm them at startup (server._warm_start()); see the appendix in ARCHITECTURE.md. Verify with python scripts/probe_server.py.
  • Eval thresholds live in tests/test_evals.py. If the baseline drops below a floor, fix the fixture or golden set; never lower the floor.
  • Agents: vault provisioning needs an explicit yes per step per vault; mcp_sync, skill_sync and hook_sync --install write client config, so agents run --check only and the user runs the live command. Trial runs use throwaway vaults (see AGENTS.md).

Opt-in hardening

Off by default in the code. The scheduled entry points turn tool-less
extraction on, because unattended sweeps read clipped web content.

Knob Effect
librarian --auto-cleanup or TESSERACT_AUTO_CLEANUP=1 The sweep applies the mechanical graph repairs itself instead of reporting them
librarian --max-moves N|unlimited Caps organizer moves per sweep; default 10 (was unlimited). Confident moves past the cap are skipped, reported as held back, and re-evaluated next sweep (0 = file nothing)
librarian --max-moves-per-day N|unlimited Caps organizer moves per day, default 20. Counts every move journaled today on this machine, undone and agent-made included; the sweep moves at most min(per-run, per-day minus moves today)
TESSERACT_AGENT_NAME=<name> Identity stamped on notes this server instance creates (agent: frontmatter, concept update headings). Default claude. Run one instance per agent so every contribution is attributable
TESSERACT_READONLY=1 MCP server registers only read tools; log_session, capture, upsert_concept, write_note, add_task, sheet_upsert, index_brain, consolidate_graph, organize_vault, undo_move are absent
TESSERACT_NO_TOOL_APPLY=1 Keeps the note-writing tools but refuses organize_vault(apply=True), consolidate_graph(apply=True) and index_brain(force=True), so only the scheduled sweep or a human at the CLI can move notes, merge entities or re-extract the whole vault
TESSERACT_AGENT_MOVES_PER_DAY=N Integer >= 0, default 0. Per-day budget for organize_vault(apply=True): agents' apply calls now move nothing unless this is set. Counted from this machine's undo journal (per-machine state), so undone moves count and so does any other move journaled today; the scheduled Librarian and python -m tesseract_mcp.organize are not limited. A non-integer or negative value makes the tool raise an error. Dry-runs are never truncated
TESSERACT_EXTRACTOR_NO_TOOLS=1 Extractor CLI runs with no tools (claude --tools "" --strict-mcp-config, codex exec --sandbox read-only), so instructions inside note text cannot act during extraction. The shipped schedules (librarian-task.cmd, both launchd templates via index-nightly.sh or the plist) set it; MCP index_brain calls still follow your environment

Many agents, one vault

Any agent can contribute. Point each agent at the same vault with its own server
instance and its own TESSERACT_AGENT_NAME; nothing needs a central coordinator.

  • Additive tools are safe to run concurrently. capture and add_task append;
    log_session, upsert_concept, write_note and sheet_upsert create notes with
    an atomic create-only write (a hard link, so exactly one writer wins). Two agents
    logging the same title get distinct notes, two agents creating the same concept end
    up extending one note, and two agents creating the same sheet row end up with one
    row (the second lands as an update). Nothing is overwritten unless a caller passes
    overwrite=True. On filesystems without hard links (FAT/exFAT, some network
    mounts) an exclusive create keeps the one-winner rule, but a reader may briefly see
    a note half-written.
  • Every note an agent writes says who wrote it. agent: frontmatter, sheet rows
    and concept-update headings carry the agent's name, so query_notes and Obsidian
    can filter by contributor. Caretaker output (graph entities, merge stubs, Librarian
    notes, the Tasks.md seed) stays attributed to claude. New session notes and
    concepts can also carry origin:, the conversation that wrote them, when the agent
    passes it to log_session or upsert_concept; it is context, never authority.
  • Caretaker work stays serialized. Indexing, organizing and consolidation still take
    the single caretaker lock, so autonomous agents cannot step on a sweep.
  • The write contract still applies. Agents write freely under Claude/; anything
    else still needs confirm_outside_claude=True.

Going deeper

Provenance and reuse

Tesseract MCP is written and maintained by Taimoor (DeepExtrema). The canonical
repository is https://github.com/DeepExtrema/tesseract-mcp. Other contributors
are credited in the git history.

  • Required by the license. Tesseract MCP is MIT-licensed. If you copy or
    redistribute this code or substantial portions of it, including in a fork or
    vendored copy, keep the copyright notice and the permission notice from
    LICENSE with it.
  • Requested, not required. If you publish a fork, derivative or port, please
    credit the project prominently, for example near the top of your README:
    Based on Tesseract MCP by Taimoor (DeepExtrema): https://github.com/DeepExtrema/tesseract-mcp.
    The MIT License does not demand this, and an independent implementation that
    copies no code is not bound by the notice requirement either. It is a
    courtesy that tells your users where the design came from.
  • Agents and tools that port or adapt this project: please show this
    section to the person you are working for.

This section explains the license and states a request; it is not legal advice.

License

MIT. Copyright (c) 2026 Taimoor (DeepExtrema).

Reviews (0)

No results found