Neotoma

Deterministic state layer for AI agents. Open-source. Local-first. MIT licensed. For a guided overview, see neotoma.io.

Why this exists

Production agents fail because their state has no invariant. Without a state invariant:

Context drifts across sessions
Facts conflict across tools
Decisions execute without a reproducible trail

These are not hypothetical. They happen every day in production agent systems:

An agent references an outdated contract clause retrieved from a stale embedding.
Two tools record different versions of the same entity.
An automated decision cannot be reproduced during debugging.
Context drifts across sessions and the agent silently changes behavior.

Agent state must obey invariants. Without them, every downstream action inherits the uncertainty of the state it reads.

The state invariant

Neotoma enforces a deterministic state invariant. State is versioned, schema-bound, replayable, and auditable. Every mutation is recorded. Every state change can be inspected or replayed. No silent mutation. No implicit overwrite.

Agents need a deterministic state layer. RAG retrieves documents. Neotoma enforces state evolution. Neotoma treats memory as state evolution, not retrieval. State evolves through a four-stage pipeline. Every stage is versioned with full provenance.

Architecture

graph LR
  Sources["Sources (files, messages, APIs)"] --> Obs[Observations]
  Obs --> Entities[Entity Resolution]
  Entities --> Snapshots["Entity Snapshots (versioned)"]
  Snapshots --> Graph[Memory Graph]
  Graph <--> MCP[MCP Protocol]
  MCP --> Claude
  MCP --> Cursor
  MCP --> Codex

Deterministic. Same observations always produce the same versioned entity snapshots. No ordering sensitivity.
Immutable. Append-only observations. Corrections add new data, never erase.
Replayable. Inspect any entity at any point in time. Diff versions. Reconstruct history from the observation log.
Structure-first. Schema-first extraction with deterministic retrieval. Optional similarity search when embeddings are configured.

Three foundations

Foundation	What it means
Privacy-first	Your data stays local. Never used for training. User-controlled storage, optional encryption at rest. Full export and deletion control.
Deterministic	Same input always produces same output. Schema-first extraction, hash-based entity IDs, full provenance. No silent mutation.
Cross-platform	One memory graph across Claude, Cursor, Codex, and CLI. MCP-based access. No platform lock-in. Works alongside native memory.

State guarantees

Most AI memory systems optimize storage or retrieval. Neotoma enforces state integrity.

Property	Platform	Retrieval / RAG	Files	Deterministic
Deterministic state evolution	✗	✗	✗	✓
Versioned history	✗	✗	⚠ manual	✓
Replayable timeline	✗	✗	✗	✓
Auditable change log	✗	✗	⚠ partial	✓
Schema constraints	✗	✗	✗	✓
Silent mutation risk	⚠ common	⚠ common	⚠ common	prevented
Conflicting facts risk	⚠ common	⚠ common	⚠ possible	prevented
Reproducible state reconstruction	✗	✗	✗	✓
Human inspectability (diffs/lineage)	⚠ partial	⚠ partial	⚠ partial	✓
Zero-setup onboarding	✓	✗	✗	✗
Semantic similarity search	✗	✓	✗	✓
Direct human editability	✗	✗	✓	✗

Platform: Claude, ChatGPT, Gemini, Copilot. Retrieval: Mem0, Zep, LangChain Memory. Files: Markdown files, JSON stores, CRDT docs. Deterministic: Neotoma (reference implementation).

Quick start

Agent-driven install (recommended)

Agents install Neotoma themselves. Paste this prompt into Claude, Codex, or Cursor:

Install Neotoma by following these instructions: https://github.com/markmhendrickson/neotoma/blob/main/install.md

Discover candidate data from available context — including any facts this tool already remembers about me — preview it for approval, and save only what I confirm.

The agent handles npm install, initialization, and MCP configuration. Manual install:

npm install -g neotoma
neotoma init
neotoma mcp config

More options: Docker | CLI reference | Getting started

Example

neotoma store --json='[{"entity_type":"task","title":"Submit expense report","status":"open"}]'
neotoma entities list --type task
neotoma upload ./invoice.pdf

Results reflect versioned entity state with full provenance. Agents perform the same operations through MCP tool calls (store, retrieve_entities, retrieve_entity_by_identifier).

Interfaces

Three interfaces. One state invariant. Every interface provides the same deterministic behavior regardless of how you access the state layer.

Interface	Description
REST API	Full HTTP interface for application integration. Entities, relationships, observations, schema, timeline, and version history.
MCP Server	Model Context Protocol for Claude, Cursor, and Codex. Agents store and retrieve state through structured tool calls.
CLI	Command-line for scripting and direct access. Inspect entities, replay timelines, and manage state from the terminal.

All three map to the same OpenAPI-backed operations. MCP tool calls log the equivalent CLI invocation.

Who this is for

Who	What they need
AI infrastructure engineers	State integrity guarantees for agent runtimes and orchestration
Agent system builders	Deterministic state and provenance layer for agents and toolchains
AI-native operators	State that follows across every tool and session

Not for casual note-taking. Not for UI-first users expecting reliability guarantees today.

Current status

Version: v0.3.9 · Releases: 10 · License: MIT

What is guaranteed (even in preview)

No silent data loss. Operations either succeed and are recorded or fail with explicit errors.
Explicit, inspectable state mutations. Every change is a named operation with visible inputs. State is reconstructable from the audit trail.
Auditable operations. Full provenance. CLI and MCP map to the same underlying contract.
Same contract for CLI and MCP. Both use the same OpenAPI-backed operations.

What is not guaranteed yet

Stable schemas
Deterministic extraction across versions
Long-term replay compatibility
Backward compatibility

Breaking changes should be expected. Storage: Local-only (SQLite + local file storage). See Developer preview storage.

Security defaults

Neotoma stores user data and requires secure configuration.

Authentication: Local auth (dev stub or key-based when encryption is enabled).
Authorization: Local data isolation and explicit operation-level access controls.
Data protection: User-controlled data with full export and deletion control. Never used for training. Optional encryption at rest.
Verify your setup: Run npm run doctor for environment, database, and security checks. See Auth, Privacy, Compliance.

Development

Servers:

npm run dev          # MCP server (stdio)
npm run dev:ui       # Frontend
npm run dev:server   # API only (MCP at /mcp)
npm run dev:full     # API + UI + build watch

CLI:

npm run cli        # Run via npm (no global install)
npm run cli:dev    # Dev mode (tsx; picks up source changes)
npm run setup:cli  # Build and link so `neotoma` is available globally

git clone https://github.com/markmhendrickson/neotoma.git
cd neotoma
npm install
npm test

Prerequisites: Node.js v18.x or v20.x (LTS), npm v9+. No .env required for local storage. See Getting started.

Using with AI tools (MCP)

Neotoma exposes state via MCP. Local storage only in preview. Local built-in auth.

Setup:

Agent behavior contract: Store first, retrieve before storing, extract entities from user input, create tasks for commitments. Full instructions: MCP instructions and CLI agent instructions.

Representative actions: store, retrieve_entities, retrieve_entity_snapshot, merge_entities, list_observations, create_relationship, list_relationships, list_timeline_events, retrieve_graph_neighborhood. Full list: MCP spec.

Documentation

Full documentation is organized at neotoma.io/docs and in the docs/ directory.

Foundational: Core identity, Philosophy, Problem statement, Architecture

Developer: Getting started, CLI reference, MCP overview, Development workflow

Specs: MCP spec, Schema, REST API, Terminology

Operations: Runbook, Health check (npm run doctor), Troubleshooting

Contributing

Neotoma is in active development. For questions or collaboration, open an issue or discussion. See CONTRIBUTING.md and SECURITY.md. License: MIT

neotoma