CyClaw

Offline-first, RAG-enforced, $ecure Local AI "Second Brain" (no internet required for RAG and cached Qwen7B-Instruct cached locally for RAG vault misses.)
Version 1.8.0 (agentic filesystem + SQL connectors, NeMo Guardrails layer)

What It Does

CyClaw is a personal RAG (Retrieval-Augmented Generation) backend that:

1. Answers questions exclusively from your local Markdown corpus — no internet by default
2. Enforces every safety invariant via LangGraph topology — not prompts, not config flags, not discipline
3. Maintains a persistent soul/personality layer (soul.md) with SHA-256 drift detection, atomic evolution writes, and user-gated modification
4. Falls back to Grok (xAI) only with explicit user confirmation in hybrid mode — triple-gated at config, env, and per-query level
5. Exposes both a FastAPI HTTP gateway and an MCP server for Claude Desktop / Copilot Studio integration
6. Ships optional, out-of-band operator layers for Dropbox corpus sync (sync/) and agentic GitHub context / governed local workflows (agentic/, .claude/) — never imported into the request path, now also drivable from the browser terminal via governed Sync and Agentic consoles
7. Extends the agentic layer to local data (v1.8) with an opt-in filesystem connector (agentic/fsconnect/ — scoped reads + gated writes over local/SMB shares, TOCTOU-safe) and a read-only SQL connector (agentic/sqlconnect/ — SELECT-only Postgres/MSSQL scaffold) — both disabled by default and out-of-band
8. Adds an optional NeMo Guardrails content-safety layer (v1.8, guardrails/) that soft-imports nemoguardrails and degrades to offline heuristic rails — defense-in-depth only, never a routing authority (graph topology stays the sole policy)

Zero telemetry. Binds to 127.0.0.1:8787 only. All embeddings run locally via sentence-transformers. No cloud dependency for offline operation. Reproducible containerized deployment via Docker is available, while agentic and sync features remain explicitly opt-in.

Version History

Architecture

User Query (HTTP POST /query or MCMC tool call)
         │
         ▼
    ┌─────────────────────────────────────────────────────┐
    │  gate.py  (FastAPI, 127.0.0.1:8787)                 │
    │  • Rate limit (60 req/min per IP — RUNS FIRST)      │
    │  • Injection filter (sanitizer.py, config-driven)   │
    │  • Soul init (PersonalityManager closure)           │
    │  • Telemetry kill block (before any SDK import)     │
    └──────────────────┬──────────────────────────────────┘
                       │
                       ▼
    ┌─────────────────────────────────────────────────────┐
    │  graph.py  (LangGraph 7-node State Machine)         │
    │                                                     │
    │  [ENTRY]                                            │
    │     ↓                                               │
    │  1. retrieve  (Chroma + BM25 + RRF fusion)          │
    │     ↓                                               │
    │  2. route_score  (top_score >= 0.028 RRF?)          │
    │     ├─ YES ──→ 3. local_llm (LM Studio :1234)       │
    │     └─ NO  ──→ 4. user_gate (needs_confirm=true)    │
    │                    ├─ confirmed + hybrid ──→        │
    │                    │      5. grok_fallback          │
    │                    └─ declined / offline ──→        │
    │                           6. offline_best_effort    │
    │     ↓ (all paths converge)                          │
    │  7. audit_logger (SHA-256 + PII redact → jsonl)     │
    │     ↓                                               │
    │  [END]                                              │
    └─────────────────────────────────────────────────────┘
                       │
                       ▼
    ┌─────────────────────────────────────────────────────┐
    │  HybridRetriever  (retrieval/hybrid_search.py)      │
    │  • ChromaDB  (semantic, all-MiniLM-L6-v2, 384d)    │
    │  • BM25Okapi (keyword, Porter stemming)             │
    │  • RRF fusion (k=60, equal 1.0/1.0 weighting)      │
    │  • Per-chunk provenance metadata in every result    │
    └─────────────────────────────────────────────────────┘

Five security invariants enforced by graph edges — not prompts:

API Key Setup (Soul Mutations)

CyClaw's soul mutation endpoints (/soul/propose, /soul/apply, /soul/reload, /soul/restore) require a Bearer API key. Without it they return HTTP 401 immediately — intentional fail-closed behavior.

All /soul/* endpoints — including GET /soul — require a valid Authorization: Bearer <key> token. Only /health and /query are unauthenticated.

Windows — PowerShell

Set for the current session only (cleared on terminal close):

$env:CYCLAW_API_KEY = "your-strong-local-secret"
uvicorn gate:app --host 127.0.0.1 --port 8787

Persist across sessions (writes to the current user's environment permanently):

[System.Environment]::SetEnvironmentVariable(
    "CYCLAW_API_KEY",
    "your-strong-local-secret",
    [System.EnvironmentVariableTarget]::User
)
# Restart your terminal, then launch normally:
uvicorn gate:app --host 127.0.0.1 --port 8787

Verify it is set before launching:

echo $env:CYCLAW_API_KEY

Windows — Command Prompt (cmd.exe)

set CYCLAW_API_KEY=your-strong-local-secret
uvicorn gate:app --host 127.0.0.1 --port 8787

Persist permanently (takes effect in new sessions):

setx CYCLAW_API_KEY "your-strong-local-secret"

Windows Server 2022 — System-wide (all users, requires admin)

[System.Environment]::SetEnvironmentVariable(
    "CYCLAW_API_KEY",
    "your-strong-local-secret",
    [System.EnvironmentVariableTarget]::Machine
)

Or via GUI: System Properties → Advanced → Environment Variables → System variables → New.

Linux / macOS — Bash / Zsh

Set for the current session:

export CYCLAW_API_KEY="your-strong-local-secret"
uvicorn gate:app --host 127.0.0.1 --port 8787

Persist in your shell profile (~/.bashrc, ~/.zshrc, or ~/.profile):

echo 'export CYCLAW_API_KEY="your-strong-local-secret"' >> ~/.bashrc
source ~/.bashrc
uvicorn gate:app --host 127.0.0.1 --port 8787

Persist for a systemd service (Linux server):

# /etc/systemd/system/cyclaw.service
[Unit]
Description=CyClaw RAG Gateway
After=network.target

[Service]
Type=simple
User=cyclaw
WorkingDirectory=/opt/CyClaw
Environment="CYCLAW_API_KEY=your-strong-local-secret"
Environment="GROK_API_KEY=offline-dummy-sk-123"
ExecStart=/opt/CyClaw/.venv/bin/uvicorn gate:app --host 127.0.0.1 --port 8787
Restart=on-failure

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now cyclaw

All platforms — .env file (already in .gitignore)

Create .env in the repo root:

CYCLAW_API_KEY=your-strong-local-secret
GROK_API_KEY=offline-dummy-sk-123

Load it before launching:

# Bash / Zsh
export $(grep -v '^#' .env | xargs)
uvicorn gate:app --host 127.0.0.1 --port 8787

# PowerShell
Get-Content .env | ForEach-Object {
    if ($_ -match '^([^#=][^=]*)=(.*)$') {
        [System.Environment]::SetEnvironmentVariable($Matches[1].Trim(), $Matches[2].Trim())
    }
}
uvicorn gate:app --host 127.0.0.1 --port 8787

Choosing a key value

CyClaw is loopback-only (127.0.0.1:8787) — the key never crosses a network. Still:

• Use at least 20 random characters: openssl rand -hex 20 (Linux/macOS) or [System.Web.Security.Membership]::GeneratePassword(24,4) (PowerShell)
• Do not reuse a password from elsewhere
• Do not commit the key to Git (.env is already in .gitignore)

Quick Start

Prerequisites

Install

git clone https://github.com/CGFixIT/CyClaw
cd CyClaw
python3.12 -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate

# 1) Install CPU-only torch first (CVE-2025-32434 fixed in 2.6.0; 2.12.1 is within the patched range)
pip install torch==2.12.1+cpu --index-url https://download.pytorch.org/whl/cpu

# 2) Install the rest, pinned to the verified transitive tree.
pip install -r requirements.txt -c constraints.txt

Required local prep

mkdir -p data/personality index logs
printf '# Soul\n' > data/personality/soul.md
export GROK_API_KEY=dummy

Run

python -m retrieval.indexer
uvicorn gate:app --host 127.0.0.1 --port 8787

Open / for the terminal UI and /health for readiness. The terminal exposes three operator consoles — Soul, Sync, and Agentic — the latter two calling POST /ops/sync and POST /ops/agentic (API-key gated, rate-limited, audited).

Project Structure

CyClaw/
├── gate.py
├── graph.py
├── config.yaml                 # single source of truth
├── README.md
├── Dropbox_Sync_Guide.md
├── mcp_hybrid_server.py        # retrieval-only MCP server
├── agentic/                    # out-of-band GitHub context + governed registry
│   ├── cli.py
│   ├── context.py
│   ├── gh_client.py
│   ├── registry.py
│   ├── writer.py               # stubbed write scaffold, non-executing
│   ├── fsconnect/              # (v1.8) local/SMB filesystem connector
│   │   ├── cli.py
│   │   ├── client.py           # scoped reads (fs_list/stat/read/grep)
│   │   ├── pathsafe.py         # TOCTOU-safe openat/O_NOFOLLOW security core
│   │   ├── writer.py           # gated, atomic writes (default-disabled)
│   │   └── indexer.py          # toggleable RAG-corpus indexing of the share
│   └── sqlconnect/             # (v1.8) read-only SQL scaffold (Postgres/MSSQL)
│       ├── cli.py
│       └── client.py           # SELECT-only query guard, env-only DSN
├── guardrails/                 # (v1.8) optional NeMo Guardrails layer (out-of-band)
│   ├── cli.py
│   ├── config.py
│   ├── integration.py          # soft-imports nemoguardrails; degrades gracefully
│   ├── rails.py                # offline heuristic rails (injection/soul/grounding)
│   ├── metrics.py              # separate logs/guardrails.jsonl stream (hashes only)
│   └── config/                 # NeMo config.yml + rails.co (Colang flows)
├── .claude/                    # local operator workflows and prompts
│   ├── commands/
│   ├── hooks/
│   ├── memory/
│   ├── patterns/
│   ├── rules/
│   ├── skills/
│   ├── tools/
│   └── utility-prompts/
├── retrieval/
│   ├── indexer.py
│   ├── hybrid_search.py
│   ├── embeddings.py
│   └── stemmer.py
├── llm/
│   └── client.py
├── sync/                       # optional Dropbox corpus sync
│   ├── cli.py
│   ├── runner.py
│   └── scheduler.py
├── utils/
│   ├── sanitizer.py
│   ├── logger.py
│   ├── personality.py
│   ├── health.py
│   └── ratelimit.py
├── tests/
├── docs/
├── static/
├── data/
│   ├── corpus/
│   └── personality/
└── .github/workflows/

Dropbox Corpus Sync

CyClaw includes an optional, out-of-band Dropbox sync layer that mirrors a Dropbox corpus into data/corpus/ without touching gate.py, graph.py, or the MCP request path.

Key capabilities

• rclone-backed pull sync with safety fuses (max_delete, max_transfer)
• audit logging for changed corpus files
• optional scheduler integration for Linux and Windows
• optional reindex trigger when corpus changes

Core commands

python -m sync.cli test
python -m sync.cli sync --dry-run
python -m sync.cli sync
python -m sync.cli status
python -m sync.cli schedule
python -m sync.cli unschedule

The same actions are available from the Sync Console panel in the terminal UI via POST /ops/sync (loopback-only, API-key gated, audited).

See Dropbox_Sync_Guide.md for full setup and scheduling details.

Agentic Layer (v1.6.0)

CyClaw now includes a concise, governed agentic layer for local operator workflows. It is opt-in, disabled by default, and fully out-of-band: it is never imported by gate.py, graph.py, or mcp_hybrid_server.py.

What it adds

• Read-only GitHub context through the gh CLI
• Governed local skills registry with explicit human gating
• Project workflows and operator helpers under .claude/
• Reusable local patterns for memory, commands, tools, hooks, and utility prompts

Security posture

• reads only in normal operation
• no GitHub token is stored or forwarded by CyClaw
• gh is invoked as an argv list, not via shell execution
• write behavior remains scaffolded and non-executing in the current release
• all agentic reads, refusals, and registry changes are audit logged

Enable it

agentic:
  enabled: true
  repo: "CGFixIT/CyClaw"
  mode: "read"
  writes_enabled: false
  gh_min_version: "2.40.0"
  registry_path: "data/agentic/skills_registry.json"

Main agentic commands

python -m agentic.cli status
python -m agentic.cli context --repo
python -m agentic.cli context --pr 123
python -m agentic.cli context --issue 45
python -m agentic.cli test
python -m agentic.cli propose-skill --name deploy --desc "..." --body-file s.md --reason "draft"
python -m agentic.cli apply-skill --name deploy --desc "..." --body-file s.md --reason "add deploy runbook" --confirm

The Agentic Console panel drives these from the terminal UI via POST /ops/agentic; skill-Apply stays disabled behind a 4-gate checklist (mode=write + writes_enabled + reason + --confirm) — dry-run only under shipped defaults.

.claude/ workflows and utility surfaces

The .claude/ tree is the local operator layer for guided workflows and reusable helper assets.

Key areas

• skills/ — reusable project skills / workflows
• commands/ — shortcut command entry points
• patterns/ — repeatable operating patterns
• tools/ — tool wrappers and helper definitions
• utility-prompts/ — reusable operator prompts
• memory/ — memory-oriented helpers / artifacts
• hooks/ and rules/ — local guardrails and automation boundaries

Examples from the current repo

• run / smoke-test workflows for CyClaw
• architecture, tests, logging, and speed refactor loops
• wrap-up / session-end workflows
• memory orchestration support patterns

Patterns and tool-call model

Use the agentic layer for:

• repo context gathering
• PR / issue inspection
• governed local skill proposals
• human-reviewed workflow execution support

Do not use it to bypass CyClaw's core RAG-first runtime or to inject autonomous write paths into the gateway.

Filesystem & SQL Connectors (v1.8)

v1.8 extends the agentic layer beyond GitHub to local data, for the regulated or security conscious use case where AI use is compliance heavy. Both connectors are opt-in, disabled by default, and fully out-of-band — never imported by gate.py, graph.py, or mcp_hybrid_server.py, so the five security invariants hold by construction. While disabled, their CLIs are a pure no-op (exit 0).

agentic/fsconnect/ — local / SMB filesystem connector

Scoped reads and separately-gated writes over a local or SMB file share, sharing one TOCTOU-safe security core.

• pathsafe.py security core — POSIX openat / O_NOFOLLOW handle-descent from a held root directory fd (so the root cannot be swapped under the process). Denies UNC, NTFS alternate data streams (file::$DATA), \\?\ / \\.\ device paths, .. traversal, and any symlink / reparse point. Segment-aware containment closes CVE-2025-53110 (sibling-prefix) and realpath + O_NOFOLLOW close CVE-2025-53109 (symlink/junction escape).
• Reads (fs_list / fs_stat / fs_read / fs_grep) confined to allowed_roots, audited, with a 5 MiB read cap and advisory OWASP∪banned_patterns content scanning.
• Writes (fs_write / fs_append / fs_mkdir / fs_move) — fully built but writes_enabled: false by default; confined to a separate writable_roots list; gated by a human reason + --confirm (for destructive ops); atomic (tmp + os.replace); content-agnostic (never calls the LLM — an operator pipes local-LLM/QWEN output in). A code-level FS_WRITE_HARD_DISABLE kill switch forces dry-run regardless of config.
• Toggleable RAG-corpus indexing of the share (index_enabled, dry-run default) stages eligible files into the corpus and triggers a reindex subprocess — enabling a generate → write → index loop without importing the retrieval layer.

python -m agentic.fsconnect.cli status
python -m agentic.fsconnect.cli read  <path>            # scoped read
python -m agentic.fsconnect.cli grep  <path> <pattern>
python -m agentic.fsconnect.cli write <path> --reason "..."   # dry-run unless writes_enabled
python -m agentic.fsconnect.cli index --apply           # stage share → corpus
python -m agentic.fsconnect.cli test                    # pre-flight self-test

Enable in config.yaml:

fsconnect:
  enabled: true
  allowed_roots: ["/srv/share"]   # REQUIRED when enabled; existing dirs
  max_file_bytes: 5242880         # 5 MiB read cap
  writes_enabled: false           # master write switch (dry-run plans while false)
  writable_roots: [null]          # null => OS default (/var/lib/cyclaw-fs | C:\CyClaw-FS)
  max_write_bytes: 10485760       # 10 MiB write cap
  index_enabled: false            # toggle RAG-corpus indexing of the share

agentic/sqlconnect/ — read-only SQL connector (v0.1 scaffold)

A disabled-by-default scaffold for read-only on-prem SQL (Postgres / MSSQL). Read-only is enforced three ways: a SELECT/WITH-only query guard (rejects DDL/DML, stacked statements, and comment-hidden keywords by scanning a quote-stripped copy), a session-level read-only transaction, and a hard allow_write: false. The DSN is read from an environment variable only (CYCLAW_SQL_DSN), never hardcoded; drivers (psycopg / pyodbc) are imported lazily.

python -m agentic.sqlconnect.cli status
python -m agentic.sqlconnect.cli schema                 # list table schemas (read-only)
python -m agentic.sqlconnect.cli query --table public.users   # bounded preview
python -m agentic.sqlconnect.cli test

sqlconnect:
  enabled: false
  driver: "postgres"             # "postgres" | "mssql"
  dsn_env: "CYCLAW_SQL_DSN"      # DSN from this env var only
  statement_timeout_ms: 5000
  max_rows: 1000
  allow_write: false             # reserved; v0.1 cannot write regardless

NeMo Guardrails (v1.8)

An opt-in, out-of-band content-safety layer in guardrails/. It is defense-in-depth only — never a routing authority: the LangGraph topology stays the sole source of policy. Absence of the guardrails: block, or enabled: false, is a pure no-op. It is never imported by gate.py, graph.py, or mcp_hybrid_server.py.

• nemoguardrails is an optional dependency. The layer soft-imports it and, when it is absent, degrades to offline heuristic rails that need no second LLM call:
  • input — light prompt-injection marker scan + soul/identity-mutation intent detection (the content-layer arm of the Soul-Governance invariant);
  • output — token-overlap grounding check against the retrieved context, flagging likely-ungrounded (hallucinated) answers below hallucination_threshold.
• When nemoguardrails is installed, the same Python checks back the live NeMo actions (via the Colang flows in guardrails/config/rails.co), so the offline heuristics and live rails never drift.
• Decisions are recorded to a separate metrics stream (logs/guardrails.jsonl) that stores only SHA-256 hashes — never raw text — mirroring the audit log's privacy posture.

python -m guardrails.cli status                         # config + nemoguardrails availability
python -m guardrails.cli check "your query here"        # run offline rails (no LLM/NeMo needed)
python -m guardrails.cli metrics                         # summarize the guardrail stream
python -m guardrails.cli test                            # pre-flight self-test

guardrails:
  enabled: false                 # opt-in; nothing runs while false
  engine: "openai"               # LM Studio / Ollama OpenAI-compatible endpoint
  model: "qwen2.5-7b-instruct"   # keep in sync with models.local_llm.model
  hallucination_threshold: 0.18  # token-overlap floor for the grounding rail
  metrics_path: "logs/guardrails.jsonl"   # separate from logs/audit.jsonl (hashes only)

Full design / wiring plan: docs/NeMo/later_development_guideline.md.

MCP Server

For Claude Desktop or other MCP-compatible clients:

{
  "mcpServers": {
    "cyclaw": {
      "command": "python",
      "args": ["/path/to/CyClaw/mcp_hybrid_server.py"]
    }
  }
}

The MCP server exposes a retrieval-only hybrid_search tool. It has no sampling capability and is intentionally isolated from the agentic, filesystem/SQL connector, NeMo Guardrails, and Dropbox corpus sync layers.

Security Model

Scope: CyClaw is a single-operator, loopback-bound local server. The full threat model — what the sandbox does and does not cover (no microVM by design) and why — is documented in docs/THREAT_MODEL.md.

Validation Commands

ruff check --select E,F,I,B,C4,S .
python -m tests.ci_rag_smoke
pytest tests/test_sanitizer.py tests/test_security.py tests/test_rate_limit.py tests/test_audit.py tests/test_client.py tests/test_personality.py
GROK_API_KEY=dummy pytest tests/test_agentic_*.py tests/test_fsconnect_*.py tests/test_sqlconnect_*.py tests/test_guardrails_*.py -q
python -m agentic.cli test
python -m agentic.fsconnect.cli test
python -m agentic.sqlconnect.cli test
python -m guardrails.cli test

Built by Chris Grady · cgfixit.com/linkedin