desk-mcp

English | Русский

Give your AI assistant eyes and hands on Windows.

Read tables, find controls by name, fill forms and verify the result. Use screenshots, OCR and mouse input when an app only exposes pixels.

46 MCP tools. Node.js + Windows PowerShell. No native npm modules, Python or .NET SDK to install. The UIA helper compiles at startup using Windows' built-in Add-Type.

Quick start · Usage · Tools · Configuration · Benchmarks · Русский

⚡ Live Demo: 10 Actions in 2.2 Seconds (Hands-Off)

_{24 seconds, muted. MP4 or WebM for the full size file.}

The recording uses the included Parcel Tracker app. UI Automation patterns fill fields and press controls without moving the cursor.

Quick start

Requirements: Windows 10 1809+ or Windows 11, Node.js 20+ (22/24 recommended), Git and an unlocked interactive desktop. Windows PowerShell 5.1 is included with Windows.

git clone https://github.com/DigitalDog1/desk-mcp.git
cd desk-mcp
npm ci

Add the server to your MCP client's configuration. Replace the path with the absolute path to your clone:

{
  "mcpServers": {
    "desk-mcp": {
      "command": "node",
      "args": ["C:/tools/desk-mcp/server.mjs"]
    }
  }
}

This example is for clients using mcpServers; other clients may use a different configuration format. If Node is not found, use C:/Program Files/nodejs/node.exe as command. Restart the client after configuration or code changes.

Use the clone for now: the npm registry returned 404 for desk-mcp on 2026-11-13. The previously documented npx -y desk-mcp command requires a published package.

Ask your assistant to call computer_permissions, list windows with computer_windows, and inspect the window you choose.

To reproduce the demo, run these commands in two terminals from the clone:

powershell -NoProfile -ExecutionPolicy Bypass -STA -File examples/demo-app.ps1

node examples/demo.mjs

The demo changes only its Parcel Tracker window and uses the clipboard. The recorded timings depend on the machine.

Usage

Find the window → inspect → act → verify.

• Use hwnd from computer_windows when titles change. Get a fresh handle after a window closes.
• Find a control by id (AutomationId), name, type or nameRegex. requireUnique: true refuses ambiguous or incomplete searches.
• Pass its elementId to computer_invoke, computer_set_value or computer_select. References expire when the worker restarts or the element disappears.
• Verify the resulting value or state. An input event being sent does not establish success.

For example, pass this to computer_batch:

{
  "steps": [
    { "tool": "computer_find", "args": { "title": "Parcel Tracker", "id": "searchBox", "requireUnique": true } },
    { "tool": "computer_set_value", "args": { "elementId": "${steps.0.elements.0.elementId}", "value": "ZX-9026-1147" } },
    { "tool": "computer_find", "args": { "title": "Parcel Tracker", "id": "searchBox" } }
  ]
}

Batches support up to 50 worker-tool steps, validate each step, and stop on errors unless stopOnError: false is set. Exact references preserve their value's type. Inspect allOk and each step's code; earlier actions are not rolled back. Browser tools and nested batches are not supported in a batch.

For smaller trees, use maxDepth: 4, interactiveOnly: true, compact: true. For repeated reads, use mode: auto and pass the last token as since with the same options. A stale baseline or a diff larger than the full view returns the full view. Check truncated and searchIncomplete before concluding that something is absent.

computer_set_value defaults to ValuePattern without focus. Use inputMode: type or paste for forms needing keyboard events. computer_type always types into the currently focused control. Paste uses the clipboard.

The same window with the cursor

_{Same window, same visible result, other mechanism. MP4 or WebM for the full size version.}

Coordinate input moves the real cursor. For controls that need a hover before the press, use hoverFirst: true (250 ms).

Tools

Full argument schemas are available in your MCP client.

computer_read_screen falls back from UIA to MSAA and then OCR. CDP browser tools are separate and use a dedicated Chrome/Edge profile. UIA patterns can leave the cursor untouched; pixel fallbacks and keyboard input can take focus. Check via and backend.

Configuration

Set these in the server entry's env object:

computer_launch and computer_close_window require confirm: true. The title filter is not a sandbox: untitled input, handles, element references and CDP need separate restrictions. Disable unnecessary tools and use a separate Windows session for unattended work.

Limitations and troubleshooting

• Run in an interactive Windows session. Services, locked sessions, protected windows and exclusive fullscreen games can block input or capture.
• After a UIA timeout, use screenshot/OCR or another target during cooldown. A worker restart rejects other pending calls too; re-read state before retrying an action that may already have reached the app.
• If a pixel click is ignored, try hoverFirst: true. Match scale to the screenshot; use nudge or drag on a canvas.
• computer_read_table treats the first row as a header by convention. Use headers: false to retain it as data.
• computer_selftest briefly presses Shift to check SendInput, even in dry-run. Held keys and buttons must be explicitly released.
• Tool errors include MCP isError and a code: line. Batches carry error codes per step. A condition timeout from computer_wait_element returns satisfied: false, reason: timeout.

Benchmarks

See method and reproducible results. Reading latency, response size and completed-task latency are separate measurements. No simulated screenshot agent or invented LLM delay is used.

Development

npm ci
npm run check
npm run test:unit
npm run check:docs
npm test
npm pack --dry-run

The Windows smoke test reads live windows and exercises Parcel Tracker if open. It does not type into an arbitrary active window. Selftest briefly presses Shift; browser launching is opt-in with DESK_TEST_CDP=1. Keep worker.ps1 in UTF-8 with BOM and the UIA helper compatible with C# 5.

Contributing · Architecture · Changelog · Publishing · Third-party notices

License

Apache License 2.0.