Meet Muse — a personal AI project built to understand the life you are already living.

Muse

Building a personal AI that learns how you live and work—and gets better at knowing when and how to help.
Local-first, provider-neutral, and honest about what is not built yet.

 ·  한국어

Muse is meant to be a continuing personal agent for one person's life and work—not only a
work assistant. It should remember context, coordinate personal tools, stay quiet when that
is better, and learn whether its last suggestion actually helped. We call this
Attunement: learning how to live and work well with you, not only storing facts about you.

The first proof point is Personal Continuity: helping you pick up an unfinished thread
without reconstructing it from scratch. That thread might be a project, a trip, a health
appointment, someone you meant to contact, or an article you were reading. In the first
version, you choose the thread and its related Muse items; automatic detection comes later.
The first local CLI slice now works with explicit local tasks and notes: create a thread,
link exact sources, run muse continue, then record whether it helped. Automatic detection,
timing, and observation remain later work. Work Resumption is one specialized use of it,
not Muse's whole identity.

What works today: personal memory, grounded recall, local personal stores, opt-in
ambient snapshots, pattern and interruption controls, guarded browser actions, traces,
checkpoints, and the first local Personal Continuity path. What comes next: more source
adapters, opt-in observation, and better timing. See the
product contract and implementation plan.

✨ Why Muse — five principles

Read these five and you know exactly what kind of agent this is.

1. Learns how to fit into your life.
   Facts and preferences matter, but the goal is broader: learn when to stay quiet, what
   kind of help fits, and whether it worked. The technical name for this is Attunement.
   Today Muse ships foundations for it; the complete learning loop is still a roadmap.

2. Yours — your personal layer can stay on your machine.
   Muse can use a local or cloud model. With no cloud credential it falls back to the local
   Ollama model; an available cloud key may select that provider. Set MUSE_LOCAL_ONLY=true
   when you need a strict on-device model boundary: cloud providers are then refused in
   code. Personal file-backed stores remain local by default.

3. Shows the evidence where it uses your data.
   Grounded recall and other supported personal-data paths attach real sources, lower weak
   matches, and reject invalid citations. This protection is not universal yet: a fast,
   uncited chat sentence can still slip past the citation checker. That gap is documented
   instead of hidden behind a “zero hallucinations” promise.

4. Correctable — your correction changes the next collaboration.
   Muse can reinforce strategies that work, retire inferred strategies that conflict with
   your correction, and remember vetoes. Attunement extends this discipline from answer
   content to intervention timing and form—without changing model weights or silently
   overriding explicit user rules. (muse memory, muse doctor --weaknesses)

5. Yours to act through — draft-first, never autonomous.
   Acts through your real tools (calendar, notes, tasks, reminders, the web) — but any
   send or action toward another person is draft-first and needs your explicit
   confirmation. Banking and money movement are permanently out of scope.

Attunement is the product promise. Local-first ownership, grounding, correction, and
draft-first action are the trust floor that makes it safe to pursue.

A native macOS desktop companion (a floating, voice-capable orb; on-device speech via
WhisperKit + Qwen3-TTS) is the newest surface — the same provider-neutral, grounded runtime.

⚡ See it

# Requirements: Node.js >= 22.12 (24 LTS recommended) + pnpm 10 · macOS only (Windows support planned)
pnpm install && pnpm build && pnpm test

# 30-second JARVIS demo (runs on your local default model, gemma4:12b via Ollama):
pnpm demo

The demo exercises chat with cross-turn memory, a credential-free proactive notice, the
setup diagnostic, and the Codex / Claude Desktop MCP bridge in one narrated run. Then
muse onboard walks you — one command at a time — from a fresh install to your first
private, cited answer.

The full command surface (muse --help):

muse status and muse today render entirely from your local stores — no API key
required (they fall back to a local briefing when the API server isn't running):

muse today	muse status

Daily-driver flows

# Personal Continuity — you choose the life/work thread and its exact local sources:
muse thread start "Plan a birthday" --kind life
muse thread link <thread-id> note birthday.md --role context
muse thread link <thread-id> task <task-id> --role next-step
muse continue <thread-id>
muse thread outcome <delivery-id> used

# JARVIS REPL — continuous conversation, token streaming, persona-aware (type /help):
muse chat --local --user me

# Ad-hoc summarisation over stdin:
cat note.md | muse chat --local --no-tools "한 단락으로 요약"   # gemma4:12b by default

# Real-time proactive daemon — notices address you by name, in your language:
muse proactive watch --user me --interval 60

# At-a-glance dashboard — model, persona, imminent tasks, last notice:
muse status --user me

What "JARVIS" means in Muse

Muse keeps a persistent personal model at ~/.muse/user-memory.json keyed by --user <id>.
Every REPL turn the model sees your facts (name, city, role…), preferences
(language, reply_style…), vetoes (things it must never suggest), goals, and the
current local date / time. Facts are auto-extracted from chat, taught with /remember,
or set directly with muse memory set (no-LLM path).

The same persona ships into muse proactive watch, so "Send Q3 memo due in 5 min" gets
translated through your prefs and lands as "Q3 예산 메모를 금융팀에 보내야 합니다. 지금
작성 시작할까요?" — same daemon, same model, no extra work. Muse doesn't just wrap a model
for a single call; it remembers you and shapes every future turn and every proactive notice.

🔧 Under the hood

• Model-neutral core. OpenAI, Anthropic, Google Gemini, OpenRouter, Ollama, LM Studio,
  and any OpenAI-compatible endpoint live behind a single ModelProvider adapter. The
  runtime calls the abstraction, never a vendor SDK directly. The same core drives the CLI,
  the API server, and the web UI.
• Tool & MCP first. Tools are first-class — read, write, or execute — with explicit risk
  levels, approval gates, and deterministic loop limits. 25 in-process muse.* servers ship
  built-in (eight pure-utility: time / text / math / json / url / crypto / diff /
  regex, plus the personal-domain set); external servers connect over stdio / SSE /
  streamable-HTTP. muse mcp serve runs the reverse direction — Muse itself AS a local,
  read-only MCP server (muse_recall cited grounded Q&A, knowledge_search ranked search,
  user_model_read your facts/preferences with confidence) another agent can connect to;
  see MCP server mode below.
• Personal-domain primitives. Markdown notes, a todo list, reminders, contacts, and calendar
  events across 5 providers (Local file, Local-ICS, Google Calendar, CalDAV, macOS Calendar.app) —
  plus macOS Reminders / Notes mirrors — all stored locally by default, queryable by the agent,
  editable from CLI / Web UI.
• Multi-agent orchestration. Sequential or parallel worker fan-out, an in-memory
  cross-agent message bus, per-run history with full conversation snapshots — exposed over
  HTTP and SSE.
• Messaging channels. Inbound/outbound adapters for Telegram, Discord, Slack, and LINE
  (plus local macOS desktop notifications), all routed through the same fail-closed
  channel-approval gate — a reply toward a person is draft-first, never autonomous.
• Deterministic safety. Guards are fail-close, hooks are fail-open, security lives in code
  (never in prompt instructions). Tool output is untrusted until sanitised. Risky local
  command execution is outside standard personal runtimes; use a dedicated coding tool
  outside Muse Work when that capability is needed.

apps/
  api/        Fastify API server (chat, agent specs, multi-agent, MCP, scheduler, calendar, tasks)
  cli/        terminal agent (commander + Ink TUI + setup wizards)
  web/        React UI — 13 panels (Chat, Today, Dashboard, Tasks, Reminders, Calendar,
              Notes, Memory, Messaging, Tools, Activity, Autonomy, Settings)
  desktop/    native macOS floating companion (SwiftPM)

packages/
  agent-core/    ReAct + Plan-Execute loops, guard pipeline, hook registry, model loop
  model/         ModelProvider interface + provider wire-format adapters
  tools/         tool registry, executor, sanitiser, approval path
  multi-agent/   SupervisorAgent, MultiAgentOrchestrator, message bus, history
  mcp/           MCP transports + loopback servers (notes / tasks / calendar) + NotesProvider
  calendar/      CalendarProvider abstraction + Local / ICS / Google / CalDAV / macOS adapters
  policy/        input / output guards, approval policies, adversarial red-team harness
  memory/        context trimming, summaries, user-memory store + auto-extraction hook
  observability/ tracing, latency / token-cost queries, JARVIS snapshot
  recall/        grounded-recall presentation / orchestration
  skills/        self-authored skills (author / curate / merge)
  a2a/           Muse-to-Muse swarm + council federation
  messaging/     Telegram / Discord / Slack / LINE adapters
  voice/         STT / TTS registry (local + cloud)
  browser/       real-Chrome control (opt-in, gated)
  autoconfigure/ zero-config provider / model / index resolution
  db/ scheduler/ auth/ cache/ resilience/ runtime-state/ runtime-settings/ macos/ prompts/ shared/

crates/
  runner/        Direct/test-only Rust runner artifact; never registered by standard model runtimes

What Muse will not do (boundaries)

Deliberate product boundaries, enforced in code — not TODOs:

• No money movement. Muse never connects to bank / brokerage accounts, initiates payments,
  or moves money. The blast radius is irreversible for a single-user assistant; a permanent
  boundary, not a deferral (outbound-safety.md).
• No autonomous third-party sends. Anything that transmits to another person (email, chat,
  message, web form / booking) is draft-first and you confirm the exact content before it
  leaves. The approval gate is fail-closed: deny / timeout / ambiguous recipient ⇒ nothing is sent.
• Single user, single environment. No multi-tenant accounts, no shared workspace, no RBAC.
  Identity is your local $USER.
• Vision input — one path excepted. Image attachments are serialized on local Ollama
  (muse ask --image), Anthropic, OpenAI Chat-Completions, OpenAI-compatible /
  OpenRouter, and Gemini. The only exception is the OpenAI Responses API path (text-only).
  Under explicit local-only mode, image bytes never leave the machine regardless.

🪟 Windows

Muse core runs on Windows: the CLI, the API server, grounded recall, and the
local Ollama model (Ollama for Windows).
Platform behavior is gated in CI on windows-latest; macOS-only integrations
(Apple Notes/Reminders mirrors, Contacts import, the desktop companion) are
disabled automatically — muse doctor shows the exact posture for your OS.

• Native actuators: set MUSE_WINDOWS_ACTUATORS=true to arm the PowerShell
  tool set — open apps/URLs, read battery/wifi/storage/frontmost window, set
  the clipboard, speak text, take screenshots, control media, and change
  volume / display sleep. Dark by default, like the macOS actuators.
• Ambient awareness: MUSE_AMBIENT_SOURCE=windows feeds the proactive daemon
  the frontmost window (clipboard strictly opt-in).
• Autostart: muse daemon --install registers a schtasks logon task
  (LaunchAgent on macOS).
• Media/volume key events are CI-verified only (no observable state on a
  runner) — report anything odd via issues.
• Voice output uses PowerShell's wav player; recording needs
  sox for Windows on PATH.
• Windows paths are CI-verified; report anything odd via issues.

🧩 Providers & configuration

Pick a model at runtime via env:

Free / offline path — Ollama with an open-source model:

brew install ollama && ollama serve &
ollama pull gemma4:12b                 # the shipped default — multimodal + grounding-strong
muse setup local                       # wires defaultModel into ~/.config/muse/config.json

See docs/setup-local-llm.md for the four tiers
(0.8B / 2B / 9B / 27B), license notes, and a latency-measuring dogfood script.

smoke:live auto-skips when Ollama is unreachable — a skip means the local runtime
isn't up, not that anything is broken.

MCP server mode (muse mcp serve)

Expose Muse itself as a local MCP server so another agent (Claude Code, Cursor, Codex, …)
can call it: your grounded, cited notes recall and the facts/preferences Muse has learned
about you, available as local tools to every agent you use — nothing leaves your machine.

claude mcp add muse -- muse mcp serve

Three read-only tools, no write/outbound access, no network listener (stdio only):

Running muse mcp serve is your explicit consent to expose these read tools to the
connecting client. See .claude/rules/outbound-safety.md for why write/outbound tools
aren't in scope here.

Cloud + API server (BYOK) — select a cloud provider (incompatible with MUSE_LOCAL_ONLY=true):

GEMINI_API_KEY=… MUSE_MODEL=gemini/gemini-2.0-flash MUSE_MODEL_PROVIDER_ID=gemini \
  pnpm --filter @muse/api dev
curl -X POST http://127.0.0.1:3030/api/chat -H 'content-type: application/json' \
  -d '{"message":"What time is it? Use a tool."}'
pnpm --filter @muse/web dev            # or the Web UI → http://localhost:5173

Set up calendar providers interactively with muse setup calendar (multi-select
Local / Local-ICS / Google / CalDAV / macOS; OAuth + app-password flows; chmod-600 credentials).

✅ Verification

Tests are the only form of verification. The repo ships these gates:

pnpm check        # build + test for every workspace (thousands of tests across all 28 packages)
pnpm smoke:broad  # 51 HTTP endpoints, diagnostic provider (no key)
pnpm smoke:live   # real LLM round-trip — LOCAL OLLAMA ONLY, gemma4:12b (auto-skips if unreachable)

smoke:live is local Ollama only by deliberate policy — it probes
${OLLAMA_BASE_URL:-http://localhost:11434} and asserts the model→tool→model loop
end-to-end across direct chat, streaming SSE, plan-execute, input guards, multi-agent
orchestration, muse.notes.search, muse.tasks.add, and muse.calendar.add. Cloud
provider keys are intentionally never consulted.

📚 Research & attributions

Muse is an independent MIT project. The designs below were studied and reimplemented from
scratch (no third-party source copied) — we record where the ideas came from, per feature.
Full notices: THIRD_PARTY_NOTICES.md.

Cross-field mechanism distillation (research discipline)

Beyond the AI-agent literature, Muse continuously mines OPEN papers from many fields —
biology, ecology, neuroscience, network science, control theory, decision & information
theory, linguistics, psychology, forensic & environmental statistics — distilling a real
mechanism into a deterministic, live-verified capability.

Each mechanism cites its paper in the module header; the verified feature inventory lives in
docs/feature-catalog/INDEX.md.

Deep dives: differentiation ·
verified feature catalog.

📖 Documentation

💬 Community & support

Questions, bugs, and feature ideas go through GitHub:

• Issues — github.com/wlsdks/Muse/issues
• Security reports — see SECURITY.md (do not open a public issue for vulnerabilities)

Contributing

This repo follows a lean-contract style for Claude Code collaboration:

• CONTRIBUTING.md — local setup, verification gates, commit / lint / test discipline
• CLAUDE.md — the contract every Claude Code agent reads first
• .claude/rules/ — domain rules (architecture, testing, commits, code style, …)
• CHANGELOG.md · SECURITY.md · CODE_OF_CONDUCT.md

Use Conventional Commits (feat:, fix:, refactor:, test:, docs:, chore:). Commits
and PR descriptions are written in English.

License

MIT. The runtime, adapters, and tooling are open source. Contributions are accepted
under the same terms — see CONTRIBUTING.md.