MemForge

Neuroscience-inspired memory system for AI agents. Sleep cycles consolidate, revise, and strengthen memories — just like biological brains.

MemForge manages agent memory across three tiers (hot → warm → cold) with vector search, a knowledge graph, LLM-driven reflection, procedural learning, and a memory revision engine that actively improves stored knowledge during idle periods.

For AI agents reading this: See CLAUDE.md for project instructions, code conventions, and architecture rules. See BACKLOG.md for open issues and improvement areas.

Project Status

Beta — Production hardening is complete. MemForge has passed 8 rounds of security audit (all clean at MEDIUM+), ships with a CI/CD pipeline, and has been benchmarked on LongMemEval (92.0% R@5 hybrid mode, 88.0% R@5 keyword mode). The full test suite covers integration paths, LLM-dependent paths via mock providers, HTTP API endpoints, and load targets.

See CONTRIBUTING.md for how to contribute and the ROADMAP.md for the long-term plan.

Why MemForge?

Most AI memory systems are passive stores — they save and retrieve, but the stored knowledge never improves. MemForge is different:

• Memories get better over time. Sleep cycles actively rewrite low-confidence memories using LLM review, producing progressively more accurate knowledge.
• The system measures its own quality. Revision stability, retrieval correlation, and contradiction rates tell you whether memory is actually improving — without external benchmarks.
• Retrieval reinforces memory. Memories that are frequently accessed and lead to good outcomes become stronger. Unused memories decay naturally.
• One database does everything. PostgreSQL handles storage, full-text search, vector similarity, and graph traversal. No Neo4j, no Pinecone, no separate systems to manage.

See INTEGRATION.md for how to wire MemForge into your agent (any framework, any language). See SPECIFICATION.md for design philosophy and ARCHITECTURE.md for internal architecture.

Features

• Tiered Memory — Hot (raw events) → Warm (consolidated, searchable, scored) → Cold (archived audit trail)
• Hybrid Search — Dual-tokenizer keyword (PostgreSQL FTS + trigram), semantic (pgvector HNSW), and asymmetric reciprocal rank fusion (semantic 1.5×) with keyword overlap boost, temporal proximity scoring, result deduplication, quality threshold, and entity detection
• Local In-Process Embeddings — EMBEDDING_PROVIDER=local uses @xenova/transformers (bge-small-en-v1.5 default) to generate embeddings in-process at ~137/sec on CPU. Zero external dependency — no Ollama or OpenAI required for semantic search.
• Query Understanding — Strips question scaffolding, auto-extracts time references as date filters, splits compound queries into independent sub-queries for multi-query retrieval
• Knowledge Graph — Entities and relationships extracted during consolidation, traversable via recursive CTEs
• Sleep Cycles — 5-phase background processor: scoring → triage → revision → graph maintenance → reflection. Includes autonomous weight adaptation.
• Memory Revision — LLM rewrites low-confidence memories (augment, correct, merge, compress)
• Reflection — LLM synthesizes higher-order insights and detects contradictions
• Meta-Reflection — Second-order reflection on reflections surfaces durable principles
• Procedural Memory — Condition→action rules extracted from reflections
• Active Ingest — Hints API, preference extraction, entity detection, supersession. Agents participate in their own memory management.
• Content Deduplication — Near-duplicate detection at ingest time prevents redundant storage
• Confidence Graduation — High-retrieval, high-feedback memories automatically strengthen
• Outcome Feedback — Structured outcome tags close the self-improvement loop
• Active Recall — Proactively surface relevant memories before agent actions
• Agent Resumption — Single endpoint returns a full context bundle for fast warm-start
• Entity Deduplication — Trigram-based duplicate entity detection and merge
• Temporal Intelligence — Time-bounded queries, decay scoring, timeline view
• Multi-Tenant — All operations scoped by agent ID
• Security Hardened — Zod validation, advisory locks, prompt injection boundaries, RLS, SSRF prevention, security headers. 8 audit rounds, all clean at MEDIUM+.
• MCP Server — 17 tools for Claude Code, Cursor, and MCP-compatible AI tools
• TypeScript SDK — Zero-dependency HTTP client for any JS runtime
• LLM Opt-In — Post-retrieval reranking and LLM-assisted ingest available but off by default

Quick Start

Prerequisites

• Node.js >= 20
• PostgreSQL 16+ with pgvector and pg_trgm extensions
• Redis (optional — degrades gracefully if unavailable)

Install & Run

git clone https://github.com/salishforge/memforge.git
cd memforge
npm install
cp .env.example .env  # edit DATABASE_URL at minimum

# Apply database schema (fresh install)
psql "$DATABASE_URL" -f schema/schema.sql

# If upgrading from v2.1.x, apply migrations in order:
psql "$DATABASE_URL" -f schema/migration-v2.2.sql
psql "$DATABASE_URL" -f schema/migration-v2.3.sql
psql "$DATABASE_URL" -f schema/migration-v2.4.sql

npm run build
npm start
# → {"level":"info","msg":"memforge listening","port":3333}

Docker

docker compose up -d

API Reference

All /memory/* routes require a Bearer token (MEMFORGE_TOKEN env var).

Memory Operations

Knowledge Graph

Reflection & Learning

Sleep Cycle & Health

System & Admin

Query Parameters

Search (/query):

• q — Search text (required)
• limit — Max results, 1-200 (default 10)
• mode — keyword, semantic, or hybrid (default: hybrid if embeddings enabled)
• after / before — ISO 8601 time bounds
• decay — Temporal decay rate per hour
• rerank — true to enable LLM post-retrieval reranking (requires ENABLE_LLM_RERANK=true)

Timeline (/timeline):

• from / to — ISO 8601 time range
• limit — Max results, 1-500 (default 50)

Request/Response Format

All responses use { ok: true, data: ... } on success and { ok: false, error: "..." } on failure.

# Add a memory
curl -X POST http://localhost:3333/memory/agent-1/add \
  -H "Authorization: Bearer $MEMFORGE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"content": "User prefers dark mode and compact layouts"}'

# Search memories
curl "http://localhost:3333/memory/agent-1/query?q=user+preferences&mode=hybrid" \
  -H "Authorization: Bearer $MEMFORGE_TOKEN"

# Trigger sleep cycle
curl -X POST http://localhost:3333/memory/agent-1/sleep \
  -H "Authorization: Bearer $MEMFORGE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"tokenBudget": 50000}'

# Active recall before an action
curl -X POST http://localhost:3333/memory/agent-1/active-recall \
  -H "Authorization: Bearer $MEMFORGE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"context": "About to modify the user dashboard layout"}'

Architecture

                    ┌─────────┐
                    │ Hot Tier │  Raw events (write-heavy)
                    └────┬────┘
                         │ consolidate (concat or LLM summarize)
                    ┌────▼────┐
                    │Warm Tier│  Searchable, scored, embedded
                    └────┬────┘
          ┌──────────────┼──────────────┐
          │              │              │
    ┌─────▼─────┐  ┌────▼────┐  ┌──────▼──────┐
    │ Knowledge │  │Retrieval│  │  Reflection  │
    │   Graph   │  │   Log   │  │   Engine     │
    └───────────┘  └─────────┘  └──────┬───────┘
                                       │
                                ┌──────▼──────┐
                                │  Procedural  │
                                │   Memory     │
                                └──────────────┘

    Sleep Cycle (background):
    Phase 1: Score importance (SQL)
    Phase 2: Triage — evict low-value, flag for revision
    Phase 3: Revise — LLM rewrites flagged memories
    Phase 4: Graph maintenance + entity deduplication
    Phase 5: Reflection — synthesize insights

Sleep Cycles

Sleep cycles are MemForge's primary differentiator. Inspired by complementary learning systems theory, they actively improve memory quality during idle periods:

1. Scoring — Composite importance from recency, frequency, centrality, reflection count, and revision stability
2. Triage — Evict memories below importance threshold to cold tier; flag low-confidence for revision
3. Revision — LLM reviews flagged memories with context (related entities, retrieval history, neighboring memories) and decides to augment, correct, merge, compress, or leave as-is
4. Graph Maintenance — Decay stale relationship edges; deduplicate similar entities
5. Reflection — If enough revisions occurred, synthesize insights from the revised knowledge base

Each phase respects a token budget to control LLM costs.

MCP Integration

MemForge ships as an MCP server with 17 tools. Add to Claude Code:

// ~/.claude/settings.json
{
  "mcpServers": {
    "memforge": {
      "command": "npx",
      "args": ["memforge-mcp"],
      "env": {
        "MEMFORGE_URL": "http://localhost:3333",
        "MEMFORGE_TOKEN": "your-token"
      }
    }
  }
}

Available tools: memforge_add, memforge_query, memforge_timeline, memforge_entities, memforge_graph, memforge_reflect, memforge_reflections, memforge_consolidate, memforge_procedures, memforge_sleep, memforge_health, memforge_stats, memforge_feedback, memforge_meta_reflect, memforge_dedup_entities, memforge_active_recall.

TypeScript SDK

import { MemForgeClient } from '@salishforge/memforge/client';

const client = new MemForgeClient({
  baseUrl: 'http://localhost:3333',
  token: 'your-token',
});

// Store memories
await client.add('agent-1', 'User requested dark mode');
await client.add('agent-1', 'Deployed v2.3.0 to production');

// Consolidate
await client.consolidate('agent-1', 'summarize');

// Search
const results = await client.query('agent-1', { q: 'deployment', mode: 'hybrid' });

// Reflect
await client.reflect('agent-1');

// Sleep cycle
const sleepResult = await client.sleep('agent-1', { tokenBudget: 50000 });

// Active recall
const context = await client.activeRecall('agent-1', 'About to deploy v2.4.0');

// Feedback loop
await client.feedback('agent-1', [retrievalId], 'positive', ['task_completed']);

// Meta-reflection
await client.metaReflect('agent-1');

// Submit retrieval hints
await client.hints('agent-1', { keywords: ['dark mode', 'layout'], entities: ['dashboard'] });

// Warm-start context bundle
const resume = await client.resume('agent-1');

LLM Tool Definitions

For direct LLM function calling (without MCP):

import { tools, toOpenAITools } from '@salishforge/memforge/tools';

// Anthropic tool_use format
const response = await anthropic.messages.create({ tools, ... });

// OpenAI function calling format
const response = await openai.chat.completions.create({
  tools: toOpenAITools(), ...
});

Benchmark Results

MemForge is evaluated on LongMemEval (ICLR 2025) — a 500-question benchmark measuring session-level recall across multi-session and temporal reasoning tasks.

LongMemEval — Hybrid Mode (v2.2.0, recommended)

Latency: p50 32 ms, p95 47 ms per query (hybrid mode, local PostgreSQL).

Configuration: concat consolidation, EMBEDDING_PROVIDER=local, KEYWORD_OVERLAP_BOOST=0.3, TEMPORAL_PROXIMITY_DAYS=7.

LongMemEval — Keyword Mode (v2.2.0, no embedding provider needed)

Latency: p50 39 ms, p95 50 ms per query (keyword mode, local PostgreSQL).

Configuration: concat consolidation, KEYWORD_OVERLAP_BOOST=0.3, TEMPORAL_PROXIMITY_DAYS=7. See benchmarks/RESULTS.md for full methodology.

How It Compares

MemForge hybrid mode (92.0% R@5) closes to within 4.6 points of MemPalace while running entirely on PostgreSQL with no external graph database. Keyword-only mode (88.0% R@5) runs with zero external dependencies — no embedding service required. Both modes outperform the BM25 baseline by 14–18 percentage points.

Configuration

See .env.example for the full list. Key variables:

Scheduling Sleep Cycles

Sleep cycles require external triggering — they don't run automatically. Recommended approaches:

Cron Job

# Run every 6 hours for each agent
0 */6 * * * curl -X POST http://localhost:3333/memory/agent-1/sleep \
  -H "Authorization: Bearer $MEMFORGE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"tokenBudget": 100000}'

MCP Tool (Claude Code)

Ask Claude to run a sleep cycle during conversation idle time:

"Run a sleep cycle for agent-1 to consolidate and revise memories"

Claude will call memforge_sleep via MCP. You can also use meta-reflection:

"Run a meta-reflection for agent-1 to synthesize higher-order patterns"

Idle-Detection Webhook

If your agent framework supports idle callbacks, trigger sleep on inactivity:

import { MemForgeClient } from '@salishforge/memforge/client';

const client = new MemForgeClient();

agent.onIdle(async (agentId, idleMs) => {
  if (idleMs > 5 * 60_000) { // 5 minutes idle
    await client.sleep(agentId, { tokenBudget: 50000 });
  }
});

Claude Code Hook

Add to your project's .claude/settings.json to run sleep cycles after long sessions:

{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "Task",
        "command": "curl -s -X POST http://localhost:3333/memory/$AGENT_ID/sleep -H 'Authorization: Bearer $MEMFORGE_TOKEN' -H 'Content-Type: application/json' -d '{\"tokenBudget\":50000}'"
      }
    ]
  }
}

Recommended Cadence

Redis Caching

MemForge caches hot-path queries in Redis with automatic invalidation on writes.

Cache key format: memforge:{agentId}:{sha256(query+limit)[0:12]}

All write operations (/add, /consolidate, /sleep, etc.) immediately invalidate the affected agent's cache. If Redis is unavailable, MemForge serves directly from PostgreSQL with no downtime.

Visit http://localhost:3333/admin/cache/dashboard for live monitoring.

Database

PostgreSQL with 12 tables: agents, hot_tier, warm_tier, cold_tier, consolidation_log, entities, relationships, warm_tier_entities, reflections, retrieval_log, memory_revisions, procedures.

Schema: schema/schema.sql + incremental migrations in schema/migration-*.sql.

Testing

npm test                  # All tests
npm run test:integration  # Database integration tests (requires PostgreSQL)
npm run test:cache        # Cache tests (requires Redis)
npm run type-check        # TypeScript strict mode check

See DEVELOPMENT.md for test architecture, coverage gaps, and how to write tests.

Known Limitations

• No built-in scheduler — Sleep cycles must be triggered externally (cron, MCP, webhook). This is by design — see Scheduling section above.
• Single process — No clustering or worker threads. Sleep cycles run in the main event loop. For high-throughput deployments, run separate instances for API serving and sleep cycle processing.
• LLM-dependent features require API keys — Summarize consolidation, reflection, meta-reflection, sleep cycle revision, and procedural extraction all require an LLM provider. Without one, MemForge still works as a tiered search engine with concat consolidation.
• No streaming consolidation — Large hot-tier backlogs (10K+ events) load into memory at once. See BACKLOG.md issue #6.
• Cold tier grows indefinitely — No retention policy or hard deletion. Archived memories accumulate forever.
• No unit tests for LLM paths — Integration tests cover CRUD operations but LLM-dependent codepaths (summarize, reflect, revise) need mocked provider tests. See BACKLOG.md issue #1.
• No HTTPS — Run behind a TLS-terminating reverse proxy in production. See SECURITY.md.

Project Documentation

License

MIT — see LICENSE for details.