BrainCore

Autonomous memory system for AI infrastructure.

BrainCore extracts, preserves, and retrieves operational knowledge from incidents, coding sessions, chat messages, and monitoring data, building a persistent knowledge graph that AI agents can query.

Live deployment: 26,966 facts · P50 71.6 ms · P95 85.2 ms · evidence grounding 98.52%

What It Does · Features · Quick Start · What Ships · Architecture · Benchmarks · MCP Integration · Configuration

What It Does

BrainCore watches your infrastructure and automatically:

1. Archives incidents, sessions, and artifacts with integrity checksums
2. Extracts structured facts using deterministic parsing plus LLM semantic analysis
3. Consolidates recurring patterns into actionable memories and playbooks
4. Publishes knowledge as searchable, queryable data

All knowledge is stored in PostgreSQL with pgvector, enabling 4-stream hybrid retrieval (SQL + full-text + vector + temporal) with Reciprocal Rank Fusion.

Features

• 7 data source parsers: OpsVault incidents, Claude Code sessions, Codex sessions, Discord digests, Telegram chats, Grafana alerts, PAI memory
• 4-stream hybrid retrieval: Structured SQL + FTS + vector similarity + temporal expansion, fused with RRF (k=60)
• Trust classes: deterministic, corroborated_llm, single_source_llm, human_curated
• Project scoping: Facts, memories, and episodes auto-tagged to projects via service mapping
• Quality gate: SHA256 fingerprint dedup, secret redaction, and assertion-class validation
• Local-first LLM: Uses vLLM (OpenAI-compatible) with automatic Claude CLI fallback
• Risk review queue: semantic truncation, prompt-injection heuristics, redaction hits, and high-risk fact kinds are queued for human review
• Parallel nightly pipeline: Automated archive-extract-consolidate-publish cycle with parallel extractors and health gating
• Eval framework: Gold-set benchmark with precision, recall, and evidence grounding metrics
• MCP-ready retrieval layer: Python retrieval library plus a minimal FastMCP example server for downstream tool exposure

Quick Start

The fastest honest way to try BrainCore on a fresh clone is:

1. Install Bun and Python dependencies.
2. Start PostgreSQL with pgvector.
3. Apply the schema migrations with bun src/cli.ts migrate.
4. Run the retrieval smoke benchmark to prove the repo wiring works.
5. Start the example MCP server or query the retrieval library directly.

Requirements

• Bun 1.1+
• Python 3.11+
• PostgreSQL 15+ (tested on 16)
• Docker or a local PostgreSQL instance with pgvector enabled

1. Install dependencies

bun install
python -m venv .venv
source .venv/bin/activate
pip install 'psycopg[binary]>=3.1' psycopg-pool pyyaml numpy requests pgvector pydantic 'mcp[cli]>=1.0'

2. Start PostgreSQL with pgvector

docker compose -f examples/docker-compose.yml up -d
sleep 5
cp .env.example .env
$EDITOR .env
# Optional: set BRAINCORE_EMBED_AUTH_TOKEN if your /embed endpoint requires auth.
set -a && . ./.env && set +a

3. Apply migrations

The supported launch path is bun src/cli.ts migrate.

bun src/cli.ts migrate

The migration runner connects with BRAINCORE_POSTGRES_DSN directly. It does
not require the psql binary on the application host. Applied migrations are
recorded in preserve.schema_migration; existing deployments with a complete
schema are baselined into the ledger instead of re-running data rewrites.

You can sanity-check the open-source schema shape like this:

psql "$BRAINCORE_POSTGRES_DSN" \
  -c "SELECT count(*) FROM pg_tables WHERE schemaname='preserve';"

Expected result on a fresh clone: 14-table preserve schema.

4. Run the smoke benchmark

export BRAINCORE_TEST_DSN="$BRAINCORE_POSTGRES_DSN"
python benchmarks/run_retrieval.py
python benchmarks/run_grounding.py

This auto-loads benchmarks/seed_smoke.sql if preserve.fact is empty
and writes:

• benchmarks/results/2026-04-09-retrieval.json
• benchmarks/results/2026-04-09-grounding.json

Those files are smoke-regression artifacts. They prove the pipeline is
wired correctly. They are not the same thing as the live-deployment
benchmark artifacts used for the public metrics.

5. Query BrainCore directly

From Python:

import os
from psycopg_pool import ConnectionPool
from mcp.memory_search import memory_search

pool = ConnectionPool(conninfo=os.environ["BRAINCORE_POSTGRES_DSN"])
result = memory_search(pool, query="docker restart loop", limit=5)
print(result["results"][0]["title"])

Or start the example MCP server:

python -m examples.mcp_server.server

See examples/mcp_server/README.md for
Claude Desktop and Inspector configuration.

Why It Exists

Most operational knowledge dies in one of four places:

• The incident note that never gets revisited.
• The chat thread that solved the problem but was never normalized.
• The coding session that found the root cause but never made it into a
  runbook.
• The monitoring alert that contained the first signal but not the final
  explanation.

Traditional RAG setups index documents. BrainCore tries to preserve
knowledge. That changes the design:

• Artifacts are archived before they are interpreted.
• Facts carry provenance and temporal validity windows.
• Weak facts can still be searchable without being allowed to promote
  into durable memory.
• Consolidation produces patterns and playbooks rather than just
  embeddings over raw text.

The result is a system that can answer questions like "what changed,
when, why, and how confident are we?" rather than only "which chunk of
text had similar words?"

What Ships In This Repo

What does not ship:

• A hosted control plane
• A general-purpose agent framework
• The larger downstream OpsVault MCP deployment
• A Docker image or GHCR publish flow

Architecture

BrainCore uses the observatory metaphor as a teaching scaffold, not as a
branding costume.

• An artifact is the photographic plate. BrainCore preserves it first so
  later extractors can re-read it without losing the original evidence.
• Deterministic extraction is the first look through the eyepiece: the
  obvious signals you can recover without model inference.
• Semantic extraction is the fainter second pass: useful, but never
  allowed to erase the direct observation.
• Trust classes are magnitudes. The brighter the source, the more
  aggressively BrainCore can promote it into durable memory.
• The four retrieval streams are the four axes of the fix. BrainCore
  aligns them with RRF so SQL, text, vector, and time each contribute
  without any one stream pretending to be the whole sky.

Benchmark Methodology

BrainCore keeps smoke-regression and production-corpus measurements
separate on purpose.

Production proof

These are the public headline numbers.

Smoke proof

These are regression signals, not headline claims.

Data flow

data sources
    |
    v
+-------------+     +------------------+     +------------------+
| archive      | --> | extract facts     | --> | quality gate      |
| checksum     |     | deterministic     |     | trust classes     |
| ttl state    |     | semantic fallback |     | evidence anchors  |
+-------------+     +------------------+     +------------------+
          |                         |                      |
          +-------------------------+----------------------+
                                    |
                                    v
                           +------------------+
                           | preserve schema   |
                           | facts, memories,  |
                           | entities, review  |
                           +------------------+
                                    |
                                    v
                           +------------------+
                           | hybrid retrieval  |
                           | SQL + FTS + vec   |
                           | + temporal (RRF)  |
                           +------------------+
                                    |
                                    v
                           +------------------+
                           | notes, tools,     |
                           | timelines, MCP    |
                           +------------------+

Trust classes

BrainCore uses four active trust classes in the shipped schema:

Retired facts remain queryable for provenance but do not behave like
active truth.

Real-World Example Queries

The read path is designed for operational questions, not for generic
chat summaries.

Example:

import os
from psycopg_pool import ConnectionPool
from mcp.memory_search import memory_search

pool = ConnectionPool(conninfo=os.environ["BRAINCORE_POSTGRES_DSN"])

result = memory_search(
    pool,
    query="ssl certificate renewal failure",
    type_filter="fact",
    limit=5,
)

for row in result["results"]:
    print(row["object_type"], row["title"], row["score"])

Knowledge Graph

The open-source repo ships a 14-table preserve schema. The tables are
split by lifecycle rather than by document format.

Key properties of the graph:

• Facts are temporal, not timeless.
• Evidence anchors survive re-extraction.
• Project identity is keyed through entities and service mapping, not
  by ad hoc string matching in every query.
• Retrieval can filter by scope and point in time without changing the
  underlying data model.

For a deeper design walkthrough, see
ARCHITECTURE.md.

4-Stream Hybrid Retrieval

BrainCore's retrieval library uses the same four streams on every query:

1. Structured SQL over entities and active facts.
2. Full-text search over facts, memories, segments, and episodes.
3. Vector similarity when an embedding service is configured.
4. Temporal and relation expansion to enrich the candidate set.

These are fused through Reciprocal Rank Fusion with RRF_K = 60.

The important design choice is that BrainCore does not force every query
through vector search. If the embedder is unavailable, the system
degrades to SQL + FTS + temporal expansion instead of failing the whole
request.

results = memory_search(
    pool,
    query="postgres replication lag root cause",
    scope="project:system-infra",
    as_of="2026-04-01T00:00:00Z",
    limit=10,
)

The response includes:

• results
• query_time_ms
• stream_counts

That makes it possible to debug retrieval behavior without guessing
which stream contributed the hit.

The 7 Data Sources

BrainCore currently ships 7 deterministic parsers.

The repo also contains extractor infrastructure files such as
deterministic.ts, semantic.ts, quality-gate.ts,
project-resolver.ts, and verify.ts, but those are framework pieces,
not source parsers themselves.

MCP Integration

BrainCore does not claim to ship a full MCP appliance. It ships the
retrieval library plus a small reference server in
examples/mcp_server/.

That example exists to prove three things:

• The retrieval library can be exposed over stdio transport.
• The repo-root mcp/ namespace collision can be handled safely.
• Pool creation can stay lazy so CI and tool introspection do not need a
  live database at import time.

The example server exposes the memory-search tool and nothing else.
That is deliberate. If your deployment needs state-at-time, timelines,
explanations, embeddings, or project-aware tools, build them on the
same retrieval library in your own server rather than pretending they
ship here already.

Start here:

• examples/mcp_server/server.py
• examples/mcp_server/README.md
• mcp/memory_search.py
• mcp/memory_models.py

The Nightly Pipeline

cron/nightly.sh is the automation backbone. It is intentionally
failure-tolerant:

• flock prevents overlap between runs.
• Archive and extraction steps can fail independently without tearing
  down the whole night.
• Post-processing happens in a clear sequence: embeddings, project
  tagging, consolidate, publish.
• Weekly and monthly maintenance are gated by date rather than hidden in
  separate jobs.
• Telegram notifications are optional, not mandatory.

The pipeline is designed to prefer partial progress over all-or-nothing
fragility.

Benchmarks

BrainCore keeps two kinds of benchmark artifacts in the repo:

• Production-corpus measurements for the launch README claims.
• Smoke-regression fixtures for CI and refactor protection.

Live deployment measurements

These values are measured against a live deployment. They are valid
README claims because they come from committed production-corpus
artifacts, not from the synthetic smoke fixture.

Smoke-regression measurements

The smoke fixture numbers are intentionally synthetic. They exist so a
change to the retrieval path can be caught quickly and reproducibly in
CI. They are not representative of live-corpus quality.

One important omission

The production retrieval artifact records relevance_at_10 = 0.0 for
the canonical 12-query set. That is not a contradiction; it is a scope
issue. The canonical queries are tuned to the synthetic sample incidents
used by the smoke fixture, while the live deployment contains a
different corpus. BrainCore therefore uses the production artifact for
facts, latency, and evidence coverage, and uses the smoke artifact only
for regression-only relevance checks.

To inspect or reproduce the benchmark artifacts, start with
benchmarks/README.md.

CLI Commands

The CLI is intentionally narrow and explicit.

Run bun src/cli.ts --help for the exact runtime usage block.

Configuration

Core configuration lives in .env.example.

The CLI treats BRAINCORE_POSTGRES_DSN as required at runtime. The
configuration module uses lazy evaluation so help screens and static
imports do not force a live environment until a command actually reads
the config.

Comparison

BrainCore is opinionated about operational knowledge rather than
chat-history memory.

BrainCore is a better fit when you care about incidents, sessions,
configuration drift, and queryable operational history more than about
remembering a conversational preference.

A Note From the Author

I built BrainCore because the most expensive operational mistakes in a
small AI system are often not the bugs themselves. They are the repeated
re-discovery cycles after the original evidence was scattered across
incident notes, terminals, dashboards, chat logs, and session history.

BrainCore is an attempt to keep those lessons in a form an agent can
query without pretending that every extracted fact deserves the same
level of trust.

Contributing

See CONTRIBUTING.md.

If you send a PR, run:

bash scripts/pre-push-gate.sh

That gate exists to keep hostnames, private paths, secret material, and
downstream project leakage out of the public repo.

Security

See SECURITY.md.

Key points:

• BrainCore redacts secrets before sending text to LLM endpoints.
• The open-source repo assumes local PostgreSQL ownership.
• Retrieval degrades when the embedder is missing; it should not fail
  closed in a way that hides operational history.

Release Hygiene

• Run bun run sanitize:check before pushing.
• The GitHub workflow runs the same sanitization gate on pushes to main and pull requests.
• Keep .env local-only and prefer localhost or authenticated LAN endpoints for /embed and retrieval sidecars.

License

BrainCore is released under the MIT License. See LICENSE.