████████╗██████╗ ██████╗ ██╗██╗      ██████╗ ████████╗
╚══██╔══╝██╔══██╗██╔══██╗██║██║     ██╔═══██╗╚══██╔══╝
   ██║   ██║  ██║██████╔╝██║██║     ██║   ██║   ██║
   ██║   ██║  ██║██╔═══╝ ██║██║     ██║   ██║   ██║
   ██║   ██████╔╝██║     ██║███████╗╚██████╔╝   ██║
   ╚═╝   ╚═════╝ ╚═╝     ╚═╝╚══════╝ ╚═════╝    ╚═╝

TDPilot Runtime v2.1.0

Name: TDPilot
Author: dreamrec

Build, debug, and verify TouchDesigner networks by talking to your AI.

TDPilot gives Claude (or Cursor, Codex, any MCP client) live control of a
running TouchDesigner session. The agent creates and wires operators, sets
real parameter values, screenshots the render to check its own work, and
can roll back anything it did — transactionally, with snapshots and a
full activity log. It knows TouchDesigner: every one of TD's operators is
covered by a reviewed knowledge atlas grounded in official Derivative docs.

Try these, verbatim, once installed:

/td-first-wow

Builds a moving feedback visual in your project, verifies it error-free, and shows you the screenshot — in about two minutes.

Make my project audio-reactive: drive the main TOP chain from the kick drum of audio_in.

Plans the CHOP analysis chain, wires it to your visuals, and proves the binding works before calling it done.

Why is my render black? Fix it.

Reads errors recursively, inspects the render chain state, finds the broken link, fixes it, and screenshots the repaired output.

Why artists trust it on real projects: every mutation can be snapshot,
validated, and rolled back; risky parameter writes are gated by an
operator-semantics contract; and the whole session is auditable. No signup,
no API key, no hosted service — your work never leaves your machine.

Install — Claude Code plugin (recommended)

Easiest: paste these two slash commands into any Claude Code session.

/plugin marketplace add dreamrec/TDPilot
/plugin install tdpilot@dreamrec-TDPilot

That installs the full MCP tool surface, the TDPilot skills (tdpilot-core, tdpilot-production, popx-touchdesigner, plus the brain skills), the brain agents, the guided slash commands (/td-first-wow, /td-audio-reactive, /td-explain-patch, /td-check, /td-snapshot), and the TD-side .tox component — one command, no Python setup required.

Shell one-liner alternative:

curl -fsSL https://raw.githubusercontent.com/dreamrec/TDPilot/main/scripts/install_claude_plugin.sh | bash

Or via npx:

npx tdpilot plugin-install

TouchDesigner side (once, after install)

Drag ~/.claude/plugins/cache/dreamrec-TDPilot/tdpilot/<version>/td_component/tdpilot.tox into your TD /local container. Or paste the auto-setup Python block from docs/INSTALL_CLAUDE_PLUGIN.md into the Textport (auto-detects the latest installed version).

Using Claude Desktop, Cursor, Codex, or another MCP client? See docs/INSTALL_OTHER_CLIENTS.md — one page per client, including the one-click .mcpb bundle for Claude Desktop. (Don't mix the Desktop and Claude Code flows on one machine.)

Documentation

Which doc do I read? Installing → INSTALL_CLAUDE_PLUGIN.md. First hour
→ GETTING_STARTED.md. Daily driving → USER_GUIDE.md. Running a show →
MANUAL.md. Looking up a tool → API_REFERENCE.md. Something broke →
TROUBLESHOOTING.md.

Install (Claude Code plugin): docs/INSTALL_CLAUDE_PLUGIN.md
Install (Claude Desktop / Cursor / Codex / generic MCP): docs/INSTALL_OTHER_CLIENTS.md
Getting started: docs/GETTING_STARTED.md
User guide: docs/USER_GUIDE.md
Memory guide: docs/MEMORY_GUIDE.md
Production manual: docs/MANUAL.md
API reference: docs/API_REFERENCE.md
Security model: docs/SECURITY.md
Troubleshooting: docs/TROUBLESHOOTING.md
Contributing: CONTRIBUTING.md
Release notes: CHANGELOG.md
Architecture deep-dives: docs/TDPILOT_CONCEPT_TO_NODE_MASTER_PLAN.md, docs/TDPILOT_EFFECTIVENESS_ROADMAP.md, docs/MCP_1_1_SURFACE.md (historical)

What This Is

A practical control layer between AI agents and TouchDesigner — built for iterative patch development, not one-shot guessing.
A planner that grounds ideas in real operators: every TD operator family (CHOP, COMP, DAT, MAT, POP, SOP, TOP) is covered by a reviewed knowledge atlas sourced from official Derivative docs, with key parameters and gotchas per operator.
A safety layer professionals can rely on: transactional apply, preflight checks, snapshots, verified rollback, parameter-semantics gating on risky writes, and an activity log of everything the agent touched.
A technique memory that learns validated patterns from your projects and reuses them — exportable, importable, shareable.
Diagnostics that close the loop: error scans, cook-time profiling, and screenshot verification so the agent proves its work instead of declaring it.

Packaged Add-ons

Reviewed operator atlas — The core local add-on for translating abstract ideas into real TD operators. Agents can use the 656-card reviewed operator atlas, Official Derivative source URLs, params, concepts, and gotchas to choose smaller, safer operator chains.
Brain skills and agents — Codex and Claude Code both ship explorer, builder, validator, recovery, and release workflows, so the same inspect -> plan -> execute -> validate discipline works in either client.
Hooks and release guards — Local deterministic checks catch plugin mirror drift, personal path leaks, stale .tox state, and unsafe release handoffs.
Optional local knowledge packs — POPX and future specialty packs stay local and user-owned; they extend planning context without adding hosted-service dependencies.

Start Here: Core Workflow

You don't need all 114 tools. Start with these and expand as needed:

Step	Tools	What You're Doing
Plan	`td_brain_plan`	Ground the user's visual intent in real TD operators, the reviewed atlas, Official Derivative docs, constraints, hints, memory, and live state
Execute	`td_brain_execute`, `td_transaction_apply`	Apply only a valid BrainPlan or PatchPlan with preflight, snapshots, rollback, and validation
Inspect	`td_get_info`, `td_get_nodes`, `td_get_params`, `td_get_errors`	Understand current state before touching anything
Check memory	`td_memory_recall`	See if a reusable technique already exists
Build	`td_create_node`, `td_connect_nodes`, `td_set_params`	Make changes in small, reversible steps
Verify	`td_get_errors`, `td_cooking_info`, `td_screenshot`	Prove the change worked
Protect	`td_snapshot_scene`, `td_restore_snapshot`	Save milestones, roll back if needed
Remember	`td_memory_learn`, `td_memory_save`	Save successful patterns for reuse

The loop: Inspect -> Plan -> Execute transactionally -> Validate -> Snapshot or Roll back -> Learn only if validated.

Everything else (vision, streaming, optimization, planning, TD2025 inspection) builds on top of this core.

Latest Release

v2.1.0 — the ultrareview cycle: host-authored planning (td_brain_ground + td_brain_propose), parameter value grounding across all 656 operators, screenshot-to-disk + motion verification, single-secret reliability, npx tdpilot update, and 114/114 tool annotations. Full history, every release: CHANGELOG.md.

Core Thinking Model (How To Think With This MCP)

Use this loop for every non-trivial task:

Inspect first — Read current state before touching anything. Start with td_get_info, td_get_nodes, td_get_node_detail, td_get_params.
Check memory — Before building from scratch, use td_memory_recall to check if a similar technique already exists in the library.
Build in small steps — Create or modify one chunk at a time. Prefer: create -> wire -> set params -> verify.
Learn and save — When you discover a reusable network pattern, use td_memory_learn to extract the recipe and td_memory_save to persist it.
Validate at the end — Always run td_get_errors on the affected root. Report warnings/errors and fix before marking done.
Control token cost — Prefer metadata checks over continuous image payloads. Ask the user before enabling high-token frame streaming.

Tool Map (114 Tools)

0) Brain Planning + Transactions

Use for non-trivial visual programming tasks where correctness, rollback, and validation matter.

td_brain_plan, td_brain_execute, td_transaction_apply, td_cockpit_render

1) Scene + Timeline + Project Lifecycle

Use for global context, playback control, save/load, and undo operations.

td_get_info, td_list_families, td_timeline, td_timeline_set, td_project_lifecycle

2) Network Build + Wiring

Use for creating, moving, renaming, connecting, and pruning structure.

td_get_nodes, td_get_node_detail, td_search_nodes
td_create_node, td_delete_node, td_copy_node, td_rename_node
td_connect_nodes, td_disconnect, td_get_connections

3) Parameters + DAT Content

Use for patch logic, expressions, config tables, scripts, and trigger pulses.

td_get_params, td_set_params, td_pulse_param
td_get_content, td_set_content, td_custom_parameters

4) Diagnostics + Capture

Use for proving behavior instead of assuming behavior.

td_screenshot, td_chop_data, td_geometry_data, td_pop_inspect
td_cooking_info, td_get_errors
td_exec_python, td_python_help, td_python_classes

Structured exec note: td_exec_python now returns JSON-safe result, result_type, and result_is_structured fields. Use it for lightweight structured probes before reaching for stdout parsing.

5) Events + Streaming

Use for reactive and continuous workflows.

td_subscribe, td_unsubscribe, td_get_events
td_capture_and_analyze
td_monitor_visual, td_stop_monitor_visual
td_stream_top, td_stop_stream_top

Token guidance: start with include_image=false for monitors/streams. Use image payloads only when visual detail is explicitly required. Prefer td_screenshot for single checks.

6) Optimization + Dynamics

Use for quality passes and temporal behavior analysis.

td_optimize_visual — now accepts direct objective_weights (e.g. {"stability": 0.8, "complexity": 0.2})
td_describe_dynamics

7) Safety + Recovery

Use for guardrails, emergency control, and rollback confidence.

td_set_param_bounds, td_clear_param_bounds
td_detect_instability, td_emergency_stabilize
td_snapshot_scene, td_list_snapshots, td_diff_snapshots, td_restore_snapshot
td_get_state_vector, td_get_timescale_state

8) Technique Memory & User Knowledge Store

Two parallel persistence surfaces — technique memory for replayable network recipes, knowledge store (new in v1.5.3) for free-form markdown reference essays (prose + math).

Technique memory — learning, saving, and replaying reusable network patterns:

td_memory_learn — Analyze a live network subtree and extract a portable recipe. Auto-detects complexity: small/medium networks get full recipes with all params and expressions; large networks get structure summaries + key params.
td_memory_save — Persist a technique to the project or global library.
td_memory_recall — Search the library by text query and/or tags. Returns summaries.
td_memory_replay — Rebuild a saved technique in a new location. Creates nodes, sets parameters and expressions, wires connections.
td_memory_list — List all saved techniques with optional filtering.
td_memory_favorite — Mark techniques as favorites and rate them (0-5).
td_memory_promote — Copy a project-level technique to the global library for use across all projects.
td_memory_export — Export the technique library as a portable JSON object for sharing or backup.
td_memory_import — Import techniques from an exported library (from td_memory_export).
td_memory_preferences — Get/set user preferences (color palettes, default resolutions, naming conventions, etc.)

User knowledge store (new in v1.5.3) — free-form markdown reference essays for prose-with-math reference content (BZ reaction equations, feedback recipes, "why this approach works" essays):

td_knowledge_save — Persist a markdown body with name/description/tags/source/notes. Project- or global-scoped. Body capped at 200 KB; split larger writeups into linked entries.
td_knowledge_recall — Search by free-text query and/or tags across name/description/tags/source/notes. Optional full_text=true also reads bodies (slower but more thorough).
td_knowledge_get — Fetch full markdown body + metadata for one entry by id.
td_knowledge_list — List entry summaries newest-first with optional filtering.

Storage lives at ~/.tdpilot/{memory,knowledge}/ with per-project and global scopes:

~/.tdpilot/
  memory/
    global/
      techniques.json
      preferences.json
    projects/{project_name}/
      techniques.json
      preferences.json
  knowledge/
    global/
      index.json
      entries/<uuid>.md
    projects/{project_name}/
      index.json
      entries/<uuid>.md

Starter technique recipes — the repo ships canonical live-visual recipes under data/techniques_starter/ (one td_memory_import-format JSON file per technique: plain feedback loop, feedback displacement bloom, audio-reactive level pulse, beat-detected flash, kaleidoscope mirror, noise-displace flow, particle GPU sprite trail, camera post chain, LFO param wobble, edge-glow composite). Load one with a single call — pass the file's parsed JSON as the data argument:

td_memory_import(data=<contents of data/techniques_starter/plain_feedback_loop.json>, scope="global")

Parameter names are verified against the operator atlas cards (a test enforces this), but every entry ships state: "candidate" with verified_on: null — they have not yet been replayed against a live TD build. Replay one with td_memory_replay, and promote it once it validates.

9. Macros & Planning (7)

Tool	Purpose
`td_create_macro`	Create a reusable macro from a template
`td_list_macros`	List available macros
`td_get_macro_params`	Get macro parameter schema
`td_plan_patch`	Plan a multi-step network patch
`td_preflight_patch`	Pre-validate a patch plan
`td_validate_recipe`	Validate a technique recipe
`td_audit_project`	Audit project subtree

10. Vision & Streaming (7)

Tool	Purpose
`td_capture_frame`	Capture a single frame from a TOP
`td_analyze_frame`	Analyze frame content (colors, regions)
`td_monitor_visual`	Start continuous visual monitoring
`td_stop_monitor_visual`	Stop visual monitoring
`td_stream_top`	Stream TOP output via WebSocket
`td_stop_stream_top`	Stop TOP streaming
`td_optimize_visual`	Get optimization suggestions for visuals

11. Knowledge Corpus (7)

Tool	Purpose
`td_search_official_docs`	Search official TD documentation
`td_get_operator_doc`	Get detailed operator documentation
`td_get_param_help`	Get parameter-level help
`td_lookup_snippets`	Find code snippets by topic
`td_lookup_palette_component`	Look up Palette component info
`td_get_release_delta`	Get changes between TD builds
`td_get_build_compatibility`	Check operator build compatibility

12. Server Introspection (3)

Tool	Purpose
`td_get_capabilities`	Report server capabilities
`td_get_server_metrics`	Get server performance metrics
`td_describe_surface`	Describe the full tool surface

13. Recommendations (3)

Tool	Purpose
`td_recommend_official_component`	Suggest official components
`td_find_official_example`	Find relevant official examples
`td_explain_better_way`	Suggest better approaches

14. TD 2025 Native (6)

Tool	Purpose
`td_python_env_status`	Inspect Python environment in TD
`td_threading_status`	Check threading configuration
`td_logger_status`	Inspect TD logger state
`td_tdresources_inspect`	Inspect TDResources categories
`td_component_standardize`	Audit/fix COMP standards
`td_color_pipeline`	Inspect color management pipeline

How To Use It (Practical Workflow)

Connect MCP client to TDPilot.
Ask for current project state.
Request a scoped patch goal.
Let agent apply changes in batches.
Require end-of-task td_get_errors check.
Save snapshot at stable milestone.
When you find something worth keeping: learn it, save it, rate it.

What It Is Good At

Building and refactoring operator networks quickly.
Inspecting modern POP systems with attribute-aware reads.
Converting high-level creative goals into concrete TD graph operations.
Audio-reactive/control-system patch scaffolding.
Automated cleanup, relayout, and consistency passes.
Diagnosing wiring/parameter/runtime errors with direct evidence.
Remembering what works and reusing it across projects.

What It Is Not Good At

Replacing artistic direction by itself.
High-level show design without iterative user feedback.
Unlimited always-on image streaming without token impact.
Ignoring TD-specific context (operator families, cook behavior, timing model).
"One shot perfect patch" generation in complex scenes.

Network Design Protocol (Default Aesthetic Rules)

When generating or reorganizing networks: use color coding by role, keep clean spacing and avoid overlaps, group nodes into functional clusters, preserve clear flow direction, name nodes by purpose, and run td_get_errors after edits.

Quick Setup

Recommended runtime (no manual Python setup in client config):

npx -y tdpilot

Local development runtime:

git clone https://github.com/dreamrec/TDPilot.git
cd TDPilot
uv sync
uv run tdpilot

TouchDesigner Side

Run the setup script once inside the TD Textport:

exec(open("/path/to/TDPilot/setup_mcp_in_td.py").read(), globals(), globals())

This installs the MCP component into /local/mcp_server by default, which means it persists across project opens within the same TD session. You only need to run this once — every project you open afterward will already have TDPilot available.

To install into a specific project instead: os.environ["TD_MCP_PARENT_PATH"] = "/project1" before running.

Alternatively, drag-and-drop td_component/tdpilot.tox into /local manually.

One-command setup helpers: macOS ./install.sh, Windows ./install.ps1

MCP Bundle (Standardized)

TDPilot ships a standard bundle in-repo:

mcp/manifest.json
mcp/profiles/claude-desktop.json, cursor.json, generic.json

Auto-generate client config:

tdpilot init --client claude-desktop
tdpilot init --client cursor --output ./cursor_mcp_config.json
tdpilot init --client generic --print-only

Doctor Command

Run a final environment/runtime check:

tdpilot doctor
tdpilot doctor --json

Environment Variables

TD_MCP_HOST (default 127.0.0.1 — supports hostnames like desktop-3lurf0p.tail88651a.ts.net)
TD_MCP_PORT (default 9981)
TD_MCP_SCHEME (default http — set to https for Tailscale HTTPS or TLS-enabled setups)
TD_MCP_WS_PORT (default 9982)
TD_MCP_TRANSPORT (stdio or streamable_http)
TD_MCP_HTTP_PORT (default 8765)
TD_MCP_CAPTURE_QUALITY (default 0.3)
TD_MCP_STREAM_MAX_FPS (default 15.0)
TD_MCP_EXEC_MODE (off, restricted, standard, full)
TDPILOT_PROJECT_NAME (set to enable per-project technique memory)
TDPILOT_MEMORY_DIR (override default ~/.tdpilot/memory/ path)

Test Suite

Run the test suite:

uv run --extra dev pytest tests/ -v

Planning quality is measured, not asserted: the brain is gated by a 50+ case concept-to-node golden eval corpus (scripts/eval_brain_golden.py) plus schema-snapshot, docs-truth, and packaging-mirror checks — all in CI.

Reliability Habit

Treat this as mandatory for every meaningful task: before edits inspect, during edits take small reversible steps, after edits run td_get_errors, before risky changes snapshot.

Community

Show what you built / ask anything — GitHub Discussions
Share a working technique — the technique-share issue template turns your patch (with real parameter values + a screenshot) into bundled TDPilot knowledge, with credit
Report a bug — issue templates (the bug form asks for tdpilot doctor --json)
Contribute — CONTRIBUTING.md; techniques and hint packs land fastest
Security — private disclosure via GHSA, see docs/SECURITY.md

License

MIT

┌─────────────────────────────────────────────────────────────────────┐
│ dreamrec // TDPilot // live laugh love                             │
└─────────────────────────────────────────────────────────────────────┘