hSUM: Stateful Understanding & Memory

Local evidence for agents.

Keep the source. Return the passage.

Install

   

_{Colored badges are clickable and go to the relevant section or page. GitHub prereleases provide a checksum-verifying, no-sudo installer plus prebuilt macOS arm64 and Linux x86_64 archives. Published cargo install and cargo binstall remain deferred while the crate has publish = false. Alpha binaries are checksummed and carry GitHub build attestations, but the macOS archive is not Apple-signed or notarized — see Verifying a release.}

Connect an agent  ·  Contributing  ·  Roadmap  ·  Status

hsum is a local-first evidence engine for coding agents. It indexes one
local source tree into immutable SQLite generations, then returns exact and
BM25-ranked passages with canonical citations through a CLI or a read-only MCP
stdio server. No account, model, daemon, telemetry, or network access is
involved after the build.

When something leaves your agent's context window, you lose it. hsum keeps
the source below the session, maps deterministic routes back to it, and returns
the passage with its citation intact.

Quickstart

# 1. Download and verify the version-pinned installer
version=0.1.0-alpha.4
base_url="https://github.com/burkan2/hSUM/releases/download/v$version"
curl -fsSLO "$base_url/install-hsum.sh"
curl -fsSLO "$base_url/SHA256SUMS"
grep ' install-hsum.sh$' SHA256SUMS | shasum -a 256 -c -

# 2. Install hSUM, register Codex, and activate this repository
cd /absolute/path/to/a/repository
bash /path/to/install-hsum.sh --activate . --confirm
# macOS alpha only: add --allow-unnotarized-alpha after reading the warning.

# 3. Search, then resolve immutable evidence
hsum search 'your identifier or "exact phrase"'
hsum get '<citation printed by search>'

# 4. Verify the effective Codex registration at any time
hsum integration status codex --json

Ask your agent

Copy this prompt into your coding agent to run the quickstart from an existing
checkout:

Install and smoke-test hSUM 0.1.0-alpha.4
(https://github.com/burkan2/hSUM) for the current Git repository. User-level
installation, user-level hSUM state, and the single Codex MCP registration are
in scope; do not modify repository files. Confirm this is macOS arm64 or Linux
x86_64. Download `install-hsum.sh` and `SHA256SUMS` from the exact
`v0.1.0-alpha.4` GitHub release, verify the installer's SHA-256 entry before
running it, and review the script. Run it with `--activate . --confirm`; on
macOS, explain that this alpha is not Apple-notarized and add
`--allow-unnotarized-alpha` only after I explicitly approve that required
acknowledgement. Verify that installation reports four read-only tools and a
successful citation round trip, and that `hsum integration status codex
--json` reports both the registration and agent policy as `current`. Do not
ask me to copy TOML, edit Codex configuration, or restart Codex. Report each
command and result, including any failure. Documentation is at
https://hsum.burkankale.com/docs/0.1.0-alpha.4/ and
https://hsum.burkankale.com/llms.txt.

init refuses dangerous roots, writes nothing into the repository, stores
the index in your user directory, and prints a search command it ran itself to
confirm the command returns evidence. Every result carries an immutable
hsum://v1/... citation that get resolves against stored bytes, not against
whatever the file looks like later.

[!NOTE]
This is an alpha. The
v0.1.0-alpha.4 GitHub prerelease
provides checksummed artifacts for macOS arm64 and Linux x86_64. The crate
remains publish = false, so cargo install is unavailable.

Connect your agent

hSUM speaks MCP over stdio only. Codex uses one user-wide registration whose
server derives an exact trusted repository from each task's working directory:

"$HSUM" integration install codex --activate . --confirm
"$HSUM" integration status codex

The install command uses Codex's supported mcp add/get interface, reads the
effective hsum entry back, launches that exact MCP command from the activated
repository, checks all four tools, runs evidence_status, and completes an
evidence_search → evidence_get citation round trip when the index is
nonempty. It also installs a bounded hSUM-managed block in Codex's user-global
AGENTS.md (or the effective AGENTS.override.md) so future agents know when
to reach for hSUM. Existing instructions are preserved, modified/duplicate
managed markers are refused, and --agent-policy skip is available as an
explicit escape hatch. The whole flow is idempotent.

Every later Codex task in the same repository gets hSUM automatically. In a
different repository the tools are still present; the first attempted use
returns REPOSITORY_NOT_ACTIVATED with the canonical root, privacy consequence,
and exact activation argv. After one repository-specific consent, the agent
runs that command and retries the same tool without restarting MCP. Repositories
retain separate indexes, trust bindings, generations, and citation namespaces.

For users who keep many repositories under one projects directory, authorize
that parent once:

"$HSUM" integration authorize-workspace codex --path ~/Projects --confirm

hSUM does not scan ~/Projects recursively or combine it into one corpus.
When a Codex task first calls hSUM from a Git repository below the authorized
directory, only that exact repository is initialized as a separate index. Home,
filesystem-root, overly broad, and Git-repository workspace scopes are refused.
Use integration revoke-workspace to stop future lazy enrollments; existing
bindings and indexes remain intact.

Repositories activated through the integration attempt one refresh before the
first evidence_search or evidence_get in each MCP task. A successful refresh
searches the newest committed generation. Busy, unsafe, partial, or
mass-deletion-refused refreshes keep the last good generation and expose the
freshness state in tool output. Manual/low-level hSUM repositories retain
explicit ingest behavior.

client config remains the low-level/manual escape hatch. Its Codex form is
workspace-dynamic; other clients remain binding-pinned:

"$HSUM" client config codex --format toml
"$HSUM" client config claude-code             # Claude Code
"$HSUM" client config claude-desktop          # Claude Desktop
"$HSUM" client config generic                 # anything MCP-capable

The server exposes exactly four read-only tools: evidence_search,
evidence_get, evidence_project, and evidence_status. It marks every
returned passage untrusted_content: true so agents treat it as evidence,
never as instructions. Privacy boundary: hSUM uploads nothing anywhere; a
cloud-backed client may forward returned passages to its own model provider
under that client's policy, and the generated config prints that warning
before the snippet.

Available in 0.1.0-alpha.4

Capability	Status	Current boundary
Local filesystem ingest	Available	One canonical root, one project, and one filesystem source per managed index
Markdown, text, and source code	Available	Lowercase .md, .markdown, .txt, .rs, .py, .ts, .tsx, .js, .jsx, .go, .java, .kt, .kts, .c, .h, .cpp, .hpp, .cc, .hh, .cxx, .rb, .cs, .swift, .php, .scala, .sh, .bash, and .sql
Exact and BM25 search	Available	auto and lexical are equivalent; there are no embeddings or vectors
Immutable citations and historical get	Available	Evidence remains resolvable while its immutable version remains in this alpha index
Atomic generations	Available	Explicit ingest; no watcher or daemon
Status and read-only doctor	Available	Diagnosis only; no repair, prune, backup, or migration command
MCP stdio	Available	One global Codex registration; every server process pins one exact trusted repository
JSONL and live connectors	Unsupported	Planned after the filesystem slice
Semantic search, models, and reranking	Unsupported	No model install or download path exists
HTTP server or web UI	Unsupported	MCP stdio is the only transport
Prebuilt installation	Available	Checksum-verifying no-sudo installer and archives for macOS arm64 and Linux x86_64; no crates.io package

Build from source

The repository pins Rust 1.91.0 with rustfmt and Clippy. Release CI runs on
clean GitHub-hosted macOS arm64 and Linux x86_64 machines. The implementation
uses Unix filesystem and locking primitives on local storage; Windows is
unsupported.

git clone <your-reviewed-copy-of-this-repository>
cd <repository>
cargo build --locked --release
./target/release/hsum --version

Ask your agent

Copy this prompt into your coding agent to install hSUM from source:

Install hSUM (https://github.com/burkan2/hSUM) from this reviewed source
checkout without changing application source or configuration files. First
confirm that the host is macOS arm64 or Linux x86_64 and check
`rustc --version` and `cargo --version`. If Rust/Cargo is missing, offer me two
choices and wait for my answer: I can install Rust from
https://www.rust-lang.org/tools/install, or you can install it with rustup
after I explicitly approve that system change. Once Cargo is available, run
`cargo build --locked --release` and `./target/release/hsum --version`. Then
give me the exact `export HSUM=...` command using this checkout's absolute path
and report the commands and results without making any other changes.
Documentation is at https://hsum.burkankale.com/docs/0.1.0-alpha.4/ and
https://hsum.burkankale.com/llms.txt.

The first build may contact Rust and crate registries to obtain the pinned
toolchain and dependencies. After a binary exists, hSUM has no network
client, account, telemetry, model download, HTTP server, or other network
runtime path. An already populated Cargo cache can build with Cargo's normal
--offline option.

For the examples below, set HSUM to the absolute path of the binary before
changing directories:

export HSUM=/absolute/path/to/checkout/target/release/hsum

First local evidence

Run the dry run first. It scans and estimates the source but writes no index,
trust binding, or repository pointer.

cd /absolute/path/to/a/repository
"$HSUM" init --dry-run
"$HSUM" init
"$HSUM" search '"HTTPServer::handle_request"' --explain

init chooses the enclosing Git root, or the current directory when there is
no enclosing Git worktree. It creates a private managed SQLite index and a
user-side trust binding. It does not modify the repository unless
--write-pointer is supplied. The optional .hsum.toml pointer is only a
logical hint; it never grants authority without a matching user trust binding.

Useful initialization controls:

"$HSUM" init --no-ingest
"$HSUM" init --index my-index --project default
"$HSUM" init --write-pointer
"$HSUM" init --index-quota-bytes 1073741824

• --no-ingest creates and trusts the index while leaving the source in
  never_succeeded state.
• --allow-broad-root is required for /, the account home, or a directory
  above the enclosing Git worktree.
• --allow-large-source raises the persisted source ceiling from 10,000 files
  or 2 GiB to the hard ceiling of 50,000 files or 8 GiB.
• --force-pointer is valid only with --write-pointer and replaces
  conflicting pointer bytes after preflight.
• Re-running init for the same trusted root is idempotent only when the stored
  index, project, source identity, and source root still match.

Command inventory

The current candidate exposes only the following command graph:

Command	Output and effect
hsum init [PATH]	Dry-run or create one managed filesystem index, project, source, and user trust binding; --rebuild safely replaces the binding and index
hsum trust PATH --confirm	Add or confirm a root-bound user trust binding for an existing matching index
hsum ingest	Dry-run or commit one authoritative filesystem generation
hsum search QUERY	Human evidence or one JSON document; exact plus BM25
hsum get CITATION_URI	Human evidence or one JSON document from an immutable citation
hsum status	Human status/problems or one JSON document
hsum doctor	Read-only human integrity diagnosis
hsum context	Human selection details or one JSON document
hsum client config CLIENT	Copyable JSON or TOML for codex, claude-code, claude-desktop, or generic
hsum client doctor CLIENT	Validate the generated configuration with a real empty-directory MCP probe
hsum help error SUBCODE	Offline explanation of one stable public error subcode
hsum completions SHELL	Completion text on stdout for bash, zsh, fish, powershell, or elvish
hsum man	A generated hsum(1) page on stdout
hsum mcp	Project-bound MCP over stdin/stdout

The only help subcommand is the offline error catalog,
hsum help error <SUBCODE>; for command help use hsum --help or
hsum <command> --help. The current candidate has no model, backup,
prune, forget,
restore, repair, watch, or daemon command.

Source discovery and ingest

The current candidate accepts valid UTF-8 text without NUL bytes. Lowercase
filename extensions are matched exactly. The per-file default is 2 MiB and
remains 2 MiB when --allow-large-source is used.

Source files chunk at declaration boundaries where the language has them —
fn, class, def, func, and their equivalents. Shell and SQL have no
unambiguous declaration keyword, so they chunk at paragraph and line
boundaries instead. Configuration formats such as .yaml, .json, and
.toml, markup such as .html and .css, and extensionless files such as
Makefile are not indexed.

Discovery stays conservative by default:

• nested .gitignore rules are honored;
• hidden names and target, node_modules, build, dist, out, vendor,
  __pycache__, and venv are excluded by default;
• .git, .ssh, .aws, .gnupg, .config, .cache, .idea, .vscode,
  environment files, common credential names, and private-key extensions are
  treated as sensitive and are not admitted by the CLI;
• source roots, directories, and files are opened without following symlinks;
• hard links are ordinary regular files and cannot be distinguished from their
  original paths, so secret-path rules are defense in depth rather than a
  content sandbox;
• file identity and metadata are checked around reads, and directory changes
  during enumeration fail closed;
• traversal is capped at 100,000 visited entries, 10,000 directories, depth
  128, and 50,000 entries in one directory.

Files that are unreadable, invalid UTF-8, contain NUL, exceed the file limit,
or change during the read become source failures rather than partially indexed
bodies. Inspect a refresh before committing it:

"$HSUM" ingest --dry-run
"$HSUM" ingest
"$HSUM" status --problems

When the indexing pipeline changes

The set of indexable extensions and the chunking rules are hashed into a
pipeline fingerprint that is stored in the index. When a new hSUM build changes
those rules, the stored fingerprint no longer matches and the index stops
opening: every command, including search and ingest, fails with
pipeline fingerprint does not match this binary.

ingest cannot recover such an index because it must open the index before it
can rebuild it. Inspect the destructive replacement first, then run it:

"$HSUM" init --rebuild --dry-run
"$HSUM" init --rebuild

Rebuild validates that the old index is structurally coherent apart from its
pipeline fingerprint, preserves other trusted roots, and creates a new index
and binding from the current source. It refuses corruption rather than
silently treating damage as compatibility. Evidence and citations from the
replaced index do not survive.

A refresh runs as two bounded passes. The first pass only enumerates and
estimates the eligible source without writing anything; a storage preflight
then checks that estimate against free capacity. The second pass re-verifies
each file's identity while spooling bodies into a bounded staging area before
the commit transaction. Because the prior generation stays fully readable until
the new one activates, managed storage transiently peaks above its steady
state during a refresh; the preflight reserve accounts for that peak.

Each successful authoritative refresh activates one atomic generation.
Unchanged bodies and immutable historical versions are reused. If every
targeted file fails, no generation or index epoch advances and the command
exits 1. A committed generation that also retains file failures exits 6
and reports a partial source. Prior evidence is carried forward where the
failure contract permits it.

Deletion has two independent confirmations:

• replacing a previously nonempty source with an empty snapshot requires
  --allow-empty-snapshot;
• deleting more than 25% of prior active documents, or more than 1,000
  documents, requires --allow-mass-delete.

Exactly 25% is allowed without the mass-delete confirmation. Use
--lock-timeout-ms 0..60000 to bound the wait for the single writer; the
default is 5,000 ms.

Search and citations

The current candidate combines three local lexical signals:

1. case-sensitive identifier-like literals;
2. conservative exact quoted spans;
3. SQLite FTS5 BM25.

The candidate lists are fused deterministically. --explain includes the
signal ranks and fixed-point fusion score. --mode auto and
--mode lexical produce the same behavior.

"$HSUM" search 'generation recovery'
"$HSUM" search '"EVIDENCE_FORGOTTEN"' --limit 20 --timeout-ms 3000 --explain
"$HSUM" search 'source_state' --json

Search limits are 1–50 results, the deadline is 100–10,000 ms, and the defaults
are 10 results and 3,000 ms. A query is at most 4,096 UTF-8 bytes. Double quotes
delimit exact spans and have no escape syntax. The CLI exposes --cursor in
the pre-stable grammar but rejects it; cursor pagination is MCP-only.

Every result includes a canonical citation:

hsum://v1/<index-id>/<source-id>/<document-id>?rev=<sha256>#bytes=<start>-<end>

The URI fixes the index, source, document, immutable revision, and exact byte
span. get resolves stored evidence, not whatever happens to be in the source
tree now:

"$HSUM" get '<citation>'
"$HSUM" get '<citation>' --max-bytes 65536 --verify-source-hash --json

get defaults to 16 KiB and allows 1–64 KiB. It may return adjacent chunks
within that bound and reports both the requested and returned citation/span.
--verify-source-hash performs a bounded live-source content check and returns
unchanged, changed, missing, blocked, or unverifiable. Normal search
uses a bounded metadata probe and labels each result
metadata_unchanged, changed_since_ingest, missing_since_ingest, or
unverifiable.

All indexed passages are emitted with untrusted_content: true. A caller must
treat a passage as evidence, never as executable instruction.

Status, context, and doctor

"$HSUM" status
"$HSUM" status --problems
"$HSUM" status --json
"$HSUM" context --json
"$HSUM" doctor

status reports generation/epoch, active document and passage counts,
per-source success or failure state, persistent quota, managed SQLite/WAL
bytes, available capacity, recovery reserve, filesystem locality, and
actionable problem codes. context shows the effective index, project, source
root, trust binding, selection origin, managed paths, and persisted quota.

doctor is read-only. It validates the SQLite application ID, exact schema
manifest and checksum, migration chain, pipeline fingerprint, WAL mode,
integrity and foreign keys, committed generation history, immutable content
digests and chunk layouts, and active FTS/literal index parity. It does not
repair data.

Storage and quota behavior

The managed index uses bundled SQLite in WAL mode with full synchronous
commits. Initialization syncs the created database and its parent directory
where the platform supports that durability operation. Generations switch
active heads inside one SQLite transaction.

The managed index directory is private by construction. Its parent is created
and required to be user-only (0700), and the database plus its WAL/SHM
sidecars must be user-only (0600), single-link regular files owned by the
current user. Every open walks the path component by component without
following symlinks (only the fixed macOS /var and /tmp system aliases are
accepted), revalidates the directory namespace against the held descriptors,
and confirms through SQLite itself that the open handle still names the
managed database file before any statement runs. A tampered, swapped, or
relocated database therefore fails closed instead of being read or repaired.

Before initialization and each ingest, hSUM estimates new managed writes and
requires free capacity for that estimate plus a recovery reserve of the larger
of 64 MiB or 10% of current managed SQLite/WAL bytes. An optional
--index-quota-bytes is stored with the source and enforced on later ingests.
Unknown estimates, arithmetic overflow, insufficient capacity, and exhausted
quota fail before generation mutation.

Known network filesystems such as NFS and SMB/CIFS are refused for managed
index storage. If locality cannot be proven, hSUM proceeds with a warning and a
status problem. Recognized iCloud Drive, Dropbox, OneDrive, Google Drive, and
Box Drive roots are reported as unsupported sync storage; the current
implementation warns rather than silently treating them as local. Put managed
data on a local filesystem with an absolute --data-dir.

Use HSUM_HOME=/absolute/path to place config, data, and cache below one root.
--data-dir and --cache-dir are per-invocation absolute overrides. Run
hsum context to see the exact resolved paths instead of assuming
platform-specific defaults.

Trust and selection

The user-side trust registry binds all of these values together:

• canonical repository root;
• random UUIDv4 binding ID;
• index name and immutable index ID;
• project name and immutable project ID.

The registry is strict, bounded, private user configuration. A copied
.hsum.toml pointer cannot authorize another root, and a copied or tampered
database must still pass identity, schema, source-cardinality, source-kind, and
source-root checks.

Direct CLI selection order is:

1. complete HSUM_INDEX and HSUM_PROJECT environment values;
2. the trust binding for the current canonical root;
3. the strict user configuration default;
4. a repository pointer, which remains only a hint and fails without trust.

Use hsum context before mutation when selection is uncertain. To bind an
existing matching managed index to a root:

"$HSUM" trust /absolute/path/to/repository --index my-index --project default --confirm

MCP ignores environment and configured defaults. It starts only from an
explicit trusted binding or a trusted current root. --binding and --project
are mutually exclusive; a project name is resolved only inside an already
trusted root context.

MCP reference

The onboarding path is in Connect your agent above;
this section is the transport reference. Generated configuration goes to
stdout only; the privacy warning prints to stderr so the snippet stays
copyable.

Generated configuration pins the binary's absolute path and one trust binding.
It does not edit client configuration files. client doctor re-parses the
exact configuration it would print, then launches that absolute executable
under the same conditions a GUI client provides: an empty working directory
with the inherited HSUM_INDEX/HSUM_PROJECT selection stripped. It
negotiates a real stdio MCP session, calls tools/list and evidence_status
against the bound project, bounds the whole probe with a deadline, and fails if
the probe process writes anything into that working directory.

The server can also be started directly:

"$HSUM" mcp --binding <uuid-v4>

It advertises exactly four hsum.api.v1 tools:

Tool	Purpose
evidence_search	Search the bound project; supports opaque, query- and snapshot-bound cursors
evidence_get	Resolve one canonical citation and optionally verify the live source hash
evidence_project	Describe the one project and its filesystem source
evidence_status	Return read-only project counts, source health, and shared index problems

Requests reject unknown fields. The transport caps each input frame at 1 MiB,
nesting at 32 levels, each structured response at 512 KiB, search passage
bodies at 256 KiB per page, and get bodies at 64 KiB. At most four blocking
tool requests run concurrently. Cancellation interrupts admitted SQLite work.
MCP stdout contains protocol frames only.

Output and errors

search, get, status, and context support --json. A successful JSON
command writes exactly one hsum.api.v1 document to stdout. Human output
escapes terminal controls, bidirectional overrides, invalid path bytes, and
other nonprinting source text.

Public failures share one catalog across human CLI, CLI JSON, and MCP. The
structured form contains:

{
  "code": "INVALID_ARGUMENT",
  "subcode": "CITATION_MALFORMED",
  "message": "the evidence citation is invalid",
  "retryable": false,
  "details": {},
  "next_action": "copy the complete citation from hsum search or evidence_search",
  "request_id": "<uuid>"
}

Errors go to stderr and leave stdout empty. Human errors render four labeled
lines: problem, cause, fix, and learn. The learn line names the offline command
hsum help error <SUBCODE> and the stable error code. That offline catalog
explains every public subcode with its category, retryability, cause, fix, and
an example, no browser or network needed. Preserve the request ID when you
report a failure.

Stable prerelease process exits:

Exit	Meaning
0	Success
1	Failure, not found, stale cursor, source-invalid, or internal error
2	Invalid arguments or confirmation required
3	Dependency/model unavailable; reserved by the shared catalog
4	Busy writer or timeout
5	Corrupt, incompatible, or integrity failure
6	A generation committed with a partial ingest
7	Permission or resource failure, including disk/quota exhaustion
130	Interrupted or cancelled

Contributor checks

The repository's local check command runs formatting, Clippy with warnings
denied, all targets/features tests, and doctests:

cargo xtask check

The equivalent individual commands are:

cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-targets --all-features
cargo test --doc --all-features

These checks are contributor evidence, not a substitute for the clean-machine
release workflow, artifact verification, documentation, and signed-tag gates.
Apple notarization and benchmark claims remain explicitly deferred. See
outputs/IMPLEMENTATION_STATUS.md for a
source-to-test evidence map and TODOS.md for the remaining work.

Verifying a release

The v0.1.0-alpha.4 archives come from a GPG-signed tag and carry a
SHA256SUMS manifest plus a GitHub SBOM attestation linking each archive to the
workflow run and commit that produced it.

# macOS example; use sha256sum -c on Linux.
asset=hsum-v0.1.0-alpha.4-aarch64-apple-darwin.zip
shasum -a 256 -c "$asset.sha256"
gh attestation verify "$asset" --repo burkan2/hSUM \
  --predicate-type https://spdx.dev/Document/v2.3

The attestation is an SPDX SBOM attestation, so --predicate-type is required;
without it gh looks for a provenance predicate and reports no matching predicate found. A successful verification prints nothing and exits 0. Add
--format json to inspect the signing workflow, tag, and commit.

The Syft SBOMs are component inventories, but their Rust package
license fields are NOASSERTION. The repository therefore records a separate,
deterministic
Cargo-declared license inventory;
the release workflow attaches that inventory and includes it in SHA256SUMS.
It is review evidence, not a legal conclusion.

The macOS archive is not Apple-signed or notarized in alpha. hSUM has no
Apple Developer account, so no Developer ID certificate exists to sign with.
Gatekeeper will refuse the binary on first launch. After verifying the checksum
and attestation above, clear the quarantine attribute yourself:

xattr -d com.apple.quarantine hsum

Only do that once the checksum and attestation both pass. Those two checks are
what establish the binary's provenance in alpha; the quarantine flag is macOS
asking for a trusted Developer ID signature and notarization hSUM cannot yet
provide, not an independent verdict on the file. Apple signing and notarization
are tracked for a later release.

License

Made by the creator of ArgonGate, Burkan Alp Kale.

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE)
• MIT license (LICENSE-MIT)

at your option. Unless you state otherwise, any contribution you submit is
dual licensed as above. Brand image provenance is recorded in
assets/brand/PROVENANCE.md.

---

The source is not the answer. It is what an answer must survive.