muser

English | 简体中文

A headless music-arrangement engine built for AI.

With muser you arrange music with data, not a mouse. Instead of an editor window, it puts a deterministic audio engine behind
a structured project format + a CLI + an MCP server: an AI agent expresses musical intent by writing or editing a project JSON,
and the engine turns it into audio.

Design principle: the engine owns the deterministic "audio truth", the AI owns "intent translation", and the interface layer is the contract between them.

Audio is powered by manguebyte (a Rust audio framework),
vendored as a git submodule.

Use it with your AI agent (no coding needed)

You don't need to know how to program. If you use an AI agent that can run commands on your computer
(Claude Code, Codex, Cursor, Kiro, WorkBuddy, …), just tell it what you want in plain words.

1. Install — say to your agent:

Install muser from GitHub (https://github.com/claw-works/muser) with its install script, then read its SKILLS.md.

2. Make music — for example:

Use muser to write me a mind-bending electronic track, about two minutes long. Render it and open it so I can listen.

Use muser to write a gentle lo-fi piano piece for studying, 90 BPM, with soft drums and vinyl crackle.

Use muser to compose a short film-trailer cue: quiet strings at first, then big drums, ending on one huge hit.

3. Change it — describe what you hear, the agent edits the project:

The drums are too loud and the middle part is boring — add a breakdown and bring the melody back at the end.

Open it in muser view so I can adjust the volumes myself.

Or have it play live and steer it as you listen:

Start a muser live session with the electronic template and keep developing it for 5 minutes; I'll tell you what to change.

4. Take it with you:

Export it as a WAV file and a MIDI file.

Update later:

Update muser to the latest release.

muser is open source and free. Every piece of the Electric Sheep suite was made this way, by agents using muser — watch it on YouTube or Bilibili.

Features

• Multi-track arrangement: drums + synths + bass, multiple clips and notes
• 100+ built-in instruments: about ten drum kits, dozens of synth presets, plus guitars, basses, organs, mallets (marimba etc.), strings, winds and choirs — all synthesized by manguebyte in real time, no system MIDI synth needed; each one is tweakable via instrument_params
• Audio tracks: put recordings, samples, loops and sound effects straight on a track ("instrument": "audio") — WAV, AIFF, FLAC, Ogg Vorbis, Opus, MP3 (plus AAC/ALAC on macOS and Windows); they go through effects, buses, sends and sidechains like any track
• Built-in SFZ sampler: sfz:/path/to.sfz (or a path relative to a sound-library folder on MUSER_SFZ_PATH) loads SFZ libraries directly
• Effect chains on tracks, buses or the master bus: reverb, delay, compressor, multiband compressor, chorus, eq, limiter, phaser, flanger, tremolo, auto-pan, stereo widener, transient shaper, distortion (with cabinet simulation), tape saturation, bitcrusher, ring modulator, frequency shifter, formant filter, granular delay and a vocoder (modulated by another track)
• Mix routing: bus groups, sends to shared reverbs/delays, nested buses, compressor sidechains (kick pumping the bass), automatic latency compensation
• Automation: instrument parameters, effect parameters, gain, pan and send levels over time (point curves / LFO / seeded random walk)
• Third-party CLAP/VST3/LV2 plugins as effects (VST) or instruments (VSTi): scan installed plugins, set parameters by name and display text, automate them
• Audition: muser audition renders a parameter at several values and reports loudness, brightness, attack and tail, so the AI can learn unfamiliar parameters
• Mix report: muser mix-report renders the song once and reports every track's, bus's and the master's loudness (dry and post-fader LUFS, peaks, level relative to the loudest track), so the AI can balance gain_db instead of guessing
• Plugin profiles: store plugin know-how (aliases, presets, audition findings) as shareable data files; plugin sounds can be saved as state files
• Built-in profiles and English skills for open-source plugins: Surge XT / Surge XT Effects, Vital (wavetable synth, with wavetables and mod matrix), Airwindows Consolidated (~470 effects, picked by name), Plasma (distortion) — install the plugin and use it by name, alias or preset
• Streaming render engine: walks the whole routing graph block by block; offline WAV rendering and render-while-playing share the same code and are sample-identical (instruments of different tracks play in parallel)
• muser view player page: playback starts as soon as the first bit is rendered; after changing volume/mute/solo the new version swaps in seamlessly at the current position; changes can be saved back to the project, and the current mix can be downloaded as a WAV, MIDI or ABC file.
  Add --device to play through this machine's sound card, with volume/mute/solo taking effect instantly (when working remotely over an SSH port forward, use the default in-browser playback)
• muser loop edit motif & loop editor: draw a motif in a piano roll (or hum it into the microphone — it is transcribed to notes without AI) or a beat in a step grid, preview it with muser's instruments, and save it to the project's loops/ or share it to your global library — agents then continue, develop or reuse these snippets (muser loop show <name> [--abc])
• muser poly edit sound editor: design poly-synth sounds (oscillators, filters, envelopes, LFOs, modulation matrix) in a local web page with instant previews, and save them to your sound library; agents put them into projects with muser poly show <name>
• Shorthand notes for agents: melodies in ABC notation ("abc"), step patterns ("grid"), a loop library of common grooves (muser loop), and a generative drummer that varies each bar and plays fills — a few lines of JSON instead of long note lists
• Humanize: timing drift, velocity and length variation, swing and push/lay-back per track or clip; a seed keeps a take you like, a new seed plays it again differently
• Arrangement layer for agents: define sections once (bars + chord progression), order them in a form (with per-pass transpose / mute / seed), and give each track parts per section; chordal patterns ("1 . 5 . 8 . 5 3", "c - - -") follow the chords with automatic voice leading, end swaps in a fill, and transform adds transpose / reverse / invert / stretch / shift / every-N / seeded variation. A full song takes about 40 lines
• muser analyze: a listening report for agents that cannot hear — per-section density, registers, harmonic rhythm, repetition, avoid notes and minor-second clashes, optional loudness and brightness, an energy curve, checks for "AI tells" (energy that never drops, instrumentation that never changes, one chord loop, fills every 4 bars, everything on the grid, identical choruses), and plain-language suggestions ("the chorus doesn't lift", "two tracks mask each other")
• MIDI export: muser midi song.muser.json writes a standard MIDI file (one track per instrument track, drums on channel 10) with exactly the notes muser plays — humanize, legato, ABC/grid/drummer expanded, sustain pedal as CC64 — for opening in a DAW or analysis tools
• ABC export: muser abc song.muser.json writes the score as standard multi-voice ABC 2.1 for models and notation tools that read ABC; --yue2 writes the native score dialect of YuE2 (Vocal/Ins melodies with chord symbols) for its full / melody modes. The key is taken from the clips' ABC or estimated, and chord symbols are estimated from the accompaniment
• Choosing sounds without ears: muser sounds list lists every sound on this machine (built-in, SFZ on MUSER_SFZ_PATH, instrument plugins); muser sounds describe renders probe notes and writes a text card (brightness, attack, sustain, release, range, velocity response, stereo width, pitch), cached locally; --keys maps a loop kit or drum kit key by key (which keys sound, loop length in bars, density). SKILLS.md teaches agents to keep what they hear in the user's own skill
• Plugin sounds and a local sound library: muser sounds presets lists a plugin's built-in sounds — VST3 .vstpresets, Surge XT patches (loaded directly), Kontakt instruments with library / category / tags; sounds that only load in the plugin's own window are captured once (by a person, or an agent with computer use) into ~/.config/muser/sound-library and used by name from then on (sounds state / capture / save / identify)
• Plugin windows (macOS): muser plugin edit opens a plugin's own editor with live audio; in muser view, 🎛 opens a sound drawer — built-in instruments and SFZ (searchable, one click), plugins with their usable sounds, or load a new one in the plugin's window, audition the track and sync the state into the project. Kontakt loop kits follow the project tempo, so tempo-locked loops work at any speed
• Per-note pitch bend ("bend": guitar bends, scoops, falls, on their own MIDI channel) and effect bypass ("bypass": true, toggled by clicking the effect in muser view)
• muser play: play a project through the sound card and exit — quick listening for the sketch-first workflow
• Two AI interfaces: CLI (shell calls) and MCP server (direct tool calls from the agent)
• Live performance (muser live): the agent writes the music a section at a time while muser plays it through this machine's sound card and never stops — when nothing new has arrived, the last section loops. Each section has its own length and time signature and can bring in new tracks, effects and automation without a gap (reverb tails carry on); you steer it in the conversation, a page shows what is playing and what is queued, and the whole session is saved as a project you can render or edit. Templates in samples/live/
• Record your own voice in muser view: record a phrase or a loop range over the backing (count-in, latency measurement), keep every take, swap takes, cut clips with scissors, drag and nudge them; the project's lyrics run as karaoke at the bottom of the screen. muser takes --timing tells an agent which phrase is early or late
• Hum lyrics: notes on choir.* can carry syllables ("lyric", or ABC w: lines); the voice sings their vowels
• Change key in one place: "transpose": 3 (or the Key − / + control in muser view) moves every pitched track without touching the notes
• A time signature per section (4/4 verse, 7/8 bridge…) and loops that run across sections ("align": "global") for polymeters
• Hand-off to a DAW: muser export-daw writes MIDI, 24-bit stems, the mix and an import guide for Live, Cubase, Logic, FL Studio and others; render --stems for stems only; render -o song.mp3 for MP3; WAV files carry section markers
• Data for visuals: muser export-data writes notes, sections, automation curves and loudness envelopes as JSON
• Install and update in one line (scripts/install.sh, install.ps1, muser self-update) and muser migrate for projects written for an older muser

Quick start

One-line install

macOS / Linux:

curl -fsSL https://raw.githubusercontent.com/claw-works/muser/main/scripts/install.sh | sh

Windows (PowerShell):

irm https://raw.githubusercontent.com/claw-works/muser/main/scripts/install.ps1 | iex

The script downloads the latest release for your system, checks it against SHA256SUMS.txt, installs it under
~/.local/share/muser/<version> (Windows: %LOCALAPPDATA%\muser\<version>) and puts muser and muser-mcp on your PATH.
The docs for agents are in ~/.local/share/muser/current/SKILLS.md. Update later with muser self-update
(muser self-update --check only looks). On Linux without ALSA (servers, Docker) it picks the headless build.

Download and run (no Rust needed)

1. Download the archive for your system from Releases:

   System	File
   macOS, Apple Silicon (M1–M4)	muser-<version>-aarch64-apple-darwin.tar.gz
   macOS, Intel	muser-<version>-x86_64-apple-darwin.tar.gz
   Windows	muser-<version>-x86_64-pc-windows-msvc.zip
   Linux desktop	muser-<version>-x86_64-unknown-linux-gnu.tar.gz (needs libasound2, present on desktop systems)
   Linux server / Docker	muser-<version>-x86_64-unknown-linux-gnu-headless.tar.gz (no sound-card playback)
   Linux arm64 desktop (e.g. Raspberry Pi 4/5, 64-bit OS)	muser-<version>-aarch64-unknown-linux-gnu.tar.gz (needs libasound2)
   Linux arm64 server / Docker (e.g. AWS Graviton)	muser-<version>-aarch64-unknown-linux-gnu-headless.tar.gz (no sound-card playback)

   The Linux builds need glibc 2.28 or newer (Amazon Linux 2023, RHEL / Rocky 8+, Debian 10+, Ubuntu 18.10+).

2. Unpack it and open a terminal in the unpacked folder. It contains muser (CLI), muser-mcp (MCP server),
   the docs for AI agents (SKILLS.md, skills/, TOOLS.md) and sample projects (samples/).

   macOS / Linux:

   tar xzf muser-<version>-aarch64-apple-darwin.tar.gz
   cd muser-<version>-aarch64-apple-darwin
   # macOS only: the binaries are not signed, so clear the download quarantine first
   xattr -dr com.apple.quarantine .
   

   Windows (PowerShell):

   Expand-Archive muser-<version>-x86_64-pc-windows-msvc.zip -DestinationPath .
   cd muser-<version>-x86_64-pc-windows-msvc
   

   If Windows SmartScreen warns about an unknown publisher, choose "More info" → "Run anyway".

3. Listen to a sample song. The player page opens in your browser and the sound comes out of this computer's sound card:

   ./muser view samples/songs/dnb-liquid-skyline.muser.json --open --device
   

   On Windows use .\muser.exe instead of ./muser. Press Ctrl+C in the terminal to stop.
   Leave out --device to play inside the browser instead (for example when muser runs on a remote machine).

4. Render it to a WAV file:

   ./muser render samples/songs/dnb-liquid-skyline.muser.json -o dnb.wav
   

5. Compose with an AI agent: open the unpacked folder in your agent (Claude Code, Codex, …) and ask it to read SKILLS.md,
   then describe the music you want. The agent writes a project JSON and renders it with ./muser;
   play the result with ./muser view <project> --open --device.
   Agents that speak MCP can use muser-mcp instead (see below).

To run muser from any folder, add the unpacked folder to your PATH.

Build from source (for developers)

1. Clone (with submodules)

muser depends on the vendored manguebyte submodule, so clone recursively:

git clone --recurse-submodules [email protected]:claw-works/muser.git
cd muser

If you already cloned without it, fetch the submodule:

git submodule update --init --recursive

2. Build

Requires Rust 1.89+ (edition 2024; the newest dependencies need it).

cargo build --release

On Linux, sound-card playback (muser view --device, enabled by default) needs the ALSA development package: sudo apt install libasound2-dev.
If you don't need it, see Build features for building without it.

The binaries are target/release/muser (CLI) and target/release/muser-mcp (MCP server).
muser in the commands below means that binary — add target/release to your PATH, or call it by full path.

Using the CLI

# Generate an example project JSON
muser new -o song.muser.json

# Validate
muser validate song.muser.json

# Render to WAV
muser render song.muser.json -o song.wav

# List instruments and effects (add a name for details, --json for machine-readable output)
muser instruments
muser instruments guitar.nylon
muser effects
muser effects reverb

# Show the signal flow (buses, sends, sidechains)
muser routing song.muser.json

# Loudness of every track/bus/master, to balance gain_db
muser mix-report song.muser.json

# For humans: local player page (timeline, volume/mute/solo re-rendered by the engine, save back to the project;
# the page refreshes when the AI edits the project)
muser view song.muser.json --open      # http://127.0.0.1:8737/
# Listening on this machine: sound comes straight out of its sound card, faders/mute/solo apply instantly, the page is a remote control
muser view song.muser.json --open --device

# Quick listen: play once through the sound card and exit (--seed N plays humanize/drummer differently)
muser play song.muser.json

# Listening report: density, registers, chords, repetition, clashes and suggestions (--audio adds loudness/brightness)
muser analyze song.muser.json --audio

# Audio files: format and length (in beats at a tempo), for writing an audio clip
muser audio vocal.wav --tempo 92

# Export: MIDI for a DAW, ABC for models and notation tools (--yue2 for YuE2's score format)
muser midi song.muser.json -o song.mid
muser abc song.muser.json -o song.abc
muser abc song.muser.json --yue2 --vocal lead --sections verse:8,chorus:8 -o score.abc

# Groove library: list, audition, and get a clip to paste into a project
muser loop list --style dnb
muser loop play dnb_two_step
muser loop show dnb_two_step --repeat 8

# Motif / loop editor (local web page): draw, preview, save to the project or share globally
muser loop edit --project song.muser.json --open
muser loop show hook --abc
muser loop hum hum.wav --tempo 96 --name hook     # hummed / whistled recording → melodic loop

# Poly-synth sound editor (local web page) and your sound library
muser poly edit --open
muser poly show my_reese          # the track JSON with the full sound embedded

# Hand-off: a folder for any DAW (MIDI, stems, mix, IMPORT.md), stems only, MP3, data for visuals
muser export-daw song.muser.json -o song-daw
muser render song.muser.json --stems stems/
muser render song.muser.json -o song.mp3
muser export-data song.muser.json -o song.json --fps 30

# Recorded vocals: list takes, check timing against the guide melody, cut / nudge clips
muser takes song.muser.json --timing
muser clip split song.muser.json --track vocal --at 32
muser clip move song.muser.json --track vocal --clip 3 --by-ms -40

# Upgrade: update muser, convert an older project
muser self-update
muser migrate old.muser.json --from 0.3.0 --write

During development you can also run cargo run -p muser-cli -- <subcommand> without installing.

Run as an MCP server (for agents)

muser-mcp          # stdio MCP server exposing 38 tools

Register this executable as a stdio server in an MCP client (such as Claude Desktop),
and the agent can arrange music with tools like create_project / add_track / add_clip / add_notes / render.

External sounds and plugins (optional)

The built-in instruments need no downloads (most samples, such as samples/songs/dnb-liquid-skyline, use only built-in sounds).
The free / open-source sound libraries and plugins below are optional: skills/ and some samples/ use them; once installed they work by name, alias or preset.

SFZ sample libraries

More free SFZ: sfzinstruments.github.io. In a project, write "instrument": "sfz:<path to .sfz>".
The samples use paths relative to the sound library (e.g. sfz:SalamanderGrandPiano-SFZ+FLAC-V3+20200602/...):
unpack the libraries into one folder and set export MUSER_SFZ_PATH=~/Music/SFZ (separate several folders with :, or ; on Windows).
muser searches these folders and their subfolders up to two levels deep; absolute paths work too.
SFZ controllers and articulations are set with cc / keyswitch in instrument_params (see skills/instruments/sfz.md).

Plugins

After installing, run muser plugin scan to check that muser finds them (system and user folders for CLAP / VST3 / LV2 are all scanned).
Plugin samples live in samples/plugins/; usage is in skills/plugins.md.

Project file format (the contract with the AI)

Time is measured in beats: 1.0 = a quarter note, independent of tempo. Minimal example:

{
  "tempo": 90.0,
  "tracks": [
    {
      "instrument": "drums.lofi",
      "clips": [
        { "start": 0.0, "length": 4.0, "notes": [
          { "pitch": 36, "start": 0.0, "length": 0.25 },
          { "pitch": 38, "start": 1.0, "length": 0.25 }
        ]}
      ]
    }
  ]
}

Drum pitches use GM drum keys (36 = kick, 38 = snare, 42 = closed hi-hat); melodic instruments use MIDI pitch (60 = middle C).

Instead of listing notes one by one, a clip can use shorthand that expands at render time, with humanize for a played feel:

{ "instrument": "poly.wurly", "humanize": { "timing_ms": 8, "drift": 0.6, "velocity": 0.07 },
  "clips": [{ "start": 0, "length": 8, "abc": "L:1/8\nK:Am\nA2 cB e2 dc | B4 z4 |" }] },
{ "instrument": "drums.lofi",
  "clips": [{ "start": 0, "length": 16, "drummer": { "loop": "lofi", "complexity": 0.5 } },
            { "start": 16, "length": 4, "grid": { "kick": "x.....x...x.....", "snare": "....x.......x...", "hat": "x.o.x.o.x.o.x.oo" } }] }

abc is ABC notation, grid a step pattern, drummer a generative drummer built on a loop; see skills/notation.md and skills/humanize.md (in Chinese).

Audio files go on audio tracks; file is relative to the project file, offset and fades are in seconds, and the audio plays at its own speed (no time-stretching). muser audio vocal.wav --tempo 92 shows its length in beats:

{ "name": "vocal", "instrument": "audio", "clips": [{ "start": 8, "length": 16, "audio": { "file": "vocal.wav", "offset": 1.5, "fade_out": 0.5 } }] }

See skills/audio.md (in Chinese).

A whole song is easier to write with the arrangement layer — sections, form, chords, chord-following parts:

"sections": { "verse": { "bars": 8, "chords": "Am7 Fmaj7 C G" }, "chorus": { "bars": 8, "chords": "F G | Em Am" } },
"form": ["verse", "chorus", "verse", { "section": "chorus", "transpose": 2 }],
"tracks": [{ "name": "bass", "instrument": "bass.finger", "parts": {
    "verse": { "chordal": "1 . . 1 . 5 8 .", "octave": 2 }, "chorus": { "chordal": "1 1 5 5 8 8 5 5", "octave": 2 } } }]

See skills/arrange.md (in Chinese) and samples/arrange/night-drive.

Build features

Both are on by default and in every release build:

• host — the plugin host (third-party CLAP/VST3/LV2 instruments and effects)
• device — sound-card playback (muser view --device; on Linux needs libasound2-dev to build)

cargo build --release --no-default-features --features muser-cli/host,muser-mcp/host builds without sound-card playback,
which is how the -headless Linux release is made.

Project layout

muser/
├─ crates/
│  ├─ muser-core/     Domain model: Project/Track/Clip/Note/Effect (all serde)
│  ├─ muser-engine/   Engine adapter: builds manguebyte instruments, streaming render (WAV / player page / sound card)
│  ├─ muser-cli/      CLI executable `muser`
│  └─ muser-mcp/      MCP server (stdio)
├─ vendor/manguebyte/ Upstream audio engine (git submodule)
├─ samples/           Sample projects (grouped: songs/instruments/params/effects/automation/routing/plugins)
├─ ARCHITECTURE.md    Architecture, layers, project JSON schema, roadmap
├─ SKILLS.md          Arrangement handbook for the AI (main entry point)
├─ skills/            Sub-skills: each instrument family, each effect, automation, routing, built-in plugin profiles
├─ profiles/          Plugin profiles shipped with muser (JSON)
├─ loops/             Groove library shipped with muser (JSON step patterns)
├─ scripts/           gen_skills.py generates skills/; render_samples.sh renders the samples in bulk
└─ TOOLS.md           CLI + MCP tool list

Further reading

The documents below are written in Chinese, except the plugin skills under skills/plugins/.

• ARCHITECTURE.md — layered architecture, data flow, project file format, vendoring strategy, roadmap
• SKILLS.md — how the AI expresses musical intent (entry point: workflow, cheat sheet, instrument and effect index)
• skills/ — parameters, scene presets and common mistakes for each instrument family and effect; automation, routing, sustain pedal
• TOOLS.md — CLI commands and MCP tools
• samples/ — grouped sample projects (songs, instruments, tuning, effects, automation, routing, plugins)

License

The code in this repository is dual-licensed under MIT / Apache-2.0. The vendored manguebyte is also MIT / Apache-2.0.