qt-commander

MCP server for Qt application introspection and automation — the
Playwright for native Qt, covering both
QWidget and QML interfaces.

Why qt-commander

AI agents (Claude, Cursor, …) can drive native Qt applications the
way Playwright drives web pages:

• No source changes — the library is injected into a running
  process; you control any Qt app (yours or a third party's) as-is.
• Both UI stacks — QWidget and QML/Qt Quick, Qt 5.15 and Qt 6.8,
  MSVC and MinGW.
• What the agent gets — full UI snapshots with geometry, z-order,
  visibility, opacity and properties; occlusion-pruned views of what a
  human actually sees; element lookup by text / type / property;
  real input pipeline clicks, typing, keyboard shortcuts, drags.
• Easy to try — uv run --project <path/to/qt-commander> qt-commander-mcp;
  the injector and library compile on demand against any detected Qt kit.

Quick Start

# Launch MCP server via uv (no global Python install needed — uv resolves
# pyproject.toml / uv.lock and manages an isolated environment; the first
# run auto-syncs the project venv and installs the qt-commander-mcp script)
uv run --project /absolute/path/to/qt-commander qt-commander-mcp

Requires uv and Python 3.10+.

MCP client configuration

Claude Code — add to .mcp.json / user-scope MCP config:

{
  "mcpServers": {
    "qt-commander": {
      "command": "uv",
      "args": ["run", "--project", "/absolute/path/to/qt-commander", "qt-commander-mcp"]
    }
  }
}

The --project flag pins the project location, so no cwd is needed —
the server starts from any working directory. (A cwd-dependent launch is a
common cause of "MCP connection failed": if the client does not honor cwd,
uv run falls back to the ambient Python and the module is not found.)

Cursor / other MCP clients — same command shape; the server speaks
stdio MCP and needs no other setup. See llms-install.md
for the full install guide (including removing legacy pip installs).

MCP Tools

Architecture

┌──────────┐     stdio      ┌──────────────┐   subprocess    ┌──────────────┐
│ AI Agent │ ◄────────────► │ MCP Server   │ ──────────────► │ qt-injector  │
└──────────┘                │ (Python)     │                 │ (C++)        │
                            └──────────────┘                 └──────┬───────┘
                                                                    │
                                                    CreateRemoteThread
                                                                    │
                                                           ┌────────▼───────┐
                                                           │ libqt-commander│
                                                           │ (C++/Qt)       │
                                                           └────────────────┘

How it works

1. AI Agent sends an MCP tool call (e.g. qt_snapshot) via stdio.
2. MCP Server spawns qt-injector.exe as a subprocess with the target PID.
3. qt-injector loads libqt-commander.dll into the target Qt process via
   CreateRemoteThread + LoadLibraryW. Before injecting the library it
   preloads the library's transitive dependency closure (Qt DLLs the
   target app does not link, e.g. Qt5Widgets for a pure QML app) from the
   library's own directory — no manual Qt DLL copies next to the target
   executable are needed. It then performs a token-authenticated handshake
   and prints the library's TCP port to stdout.
4. MCP Server connects to the library over TCP and relays RPC calls
   (snapshot, click, input, etc.) using a 4-byte length-prefix frame protocol.

Testing

Everything (C++ unit + E2E suites, pytest, and the deployment-level preload
verification) runs from one CMake build tree:

# Single build tree (injector + library + test apps + all tests).
# Both Qt5 and Qt6 are supported; pick the Qt you want to validate:
#   Qt5 MSVC: -DQT_MAJOR_VERSION=5 -DQt5_DIR=C:/Qt/5.15.2/msvc2019_64/lib/cmake/Qt5
#   Qt6 MSVC: -DQT_MAJOR_VERSION=6 -DQt6_DIR=C:/Qt/6.8.3/msvc2022_64/lib/cmake/Qt6
#   Qt6 MinGW: same, but Qt6_DIR=C:/Qt/6.8.3/mingw_64/lib/cmake/Qt6
#              with the MinGW toolchain on PATH (see "MinGW" below)
cmake -S . -B build/msvc -G Ninja ^
  -DBUILD_INJECTOR=ON -DBUILD_TESTS=ON -DWITH_QML=ON ^
  -DCMAKE_BUILD_TYPE=Release -DQT_MAJOR_VERSION=6 ^
  -DQt6_DIR=C:/Qt/6.8.3/msvc2022_64/lib/cmake/Qt6

# Build everything, then run ALL tests in one command:
cmake --build build/msvc
ctest --test-dir build/msvc --output-on-failure

verify_preload (E2E deployment checks) auto-detects the Qt major AND
toolchain kit (msvc/mingw) of the deployed libqt-commander.dll and
verifies the matching DLL set — even windeployqt follows the deployment's
kit — so the same script validates Qt5/Qt6 × msvc/mingw deployments from
either build tree.

MinGW

MinGW builds are fully supported (Qt5 and Qt6 MinGW kits). Use
qt_build with toolchain="mingw": pass the MinGW toolchain's bin dir
as vcvars_path and the kit's qtenv2.bat as qt_env (MinGW Qt kits
ship qtenv2.bat just like MSVC kits). qt_detect_msvc_and_qt reports
MinGW toolchains (mingw_toolchains) and tags each Qt kit with its
kit ("msvc"/"mingw").

Notes:

• Compiler version: Qt 5's official MinGW kit ships GCC 8.1, whose
  libstdc++ cannot compile std::filesystem headers (fixed in 8.3); use
  a GCC ≥ 9 toolchain (e.g. Qt's bundled mingw1310_64) for Qt 5 too.
  qt_build pins the compiler explicitly (-DCMAKE_C/CXX_COMPILER), so
  other gcc builds on PATH (e.g. a Strawberry Perl toolchain) never get
  picked up.
• Runtime DLLs: MinGW executables need libgcc_s_seh-1.dll,
  libstdc++-6.dll, libwinpthread-1.dll next to them. The build
  deploys the compiler's own runtime (a Qt kit's older runtime lacks
  newer symbols), and verify_preload matches the deployed app's runtime
  to the library's.
• Kit matching: the injector library, its deployment, and the target
  application must share the same Qt kit (all MSVC or all MinGW) —
  mixing kits loads two Qt module sets into one process and breaks the
  preload closure.

ctest runs 19 suites: 14 injector C++ suites (including three real E2E
injection suites against the widget test app), 3 library C++ suites, the
full pytest suite (python_unit_tests), and the E2E preload verification
(verify_preload, labeled e2e, which needs the qt_build artifacts in
.qt-commander/bin).

The E2E suites auto-anchor their working directory to their own build
tree (test_util.h::chdir_to_exe_dir), so they produce identical results
when launched directly from the repo root or through ctest.

Quick subsets:

pytest tests/ -q                                   # Python only
ctest --test-dir build/msvc -LE e2e                # skip slow E2E
ctest --test-dir build/msvc -R "test_selector"     # one suite

Test matrix (verified state, 2026-08)

Project Structure

qt-commander/
├── qt_commander/              Python MCP server
│   ├── server.py            FastMCP app, 22 tools + 2 resources
│   ├── session.py           Session/SessionManager with RPC lock
│   ├── rpc_client.py        Subprocess injector launcher
│   ├── builder.py           On-demand MSVC build orchestrator
│   ├── process_detector.py  Cross-platform Qt process discovery
│   ├── environment_detector.py  MSVC/Qt build environment auto-detection
│   ├── framing.py           4-byte BE length-prefix frame protocol
│   ├── occlusion.py         Snapshot occlusion solving (drop covered
│   │                        elements, mark visible ratio)
│   └── errors.py            MCP error code registry
│
├── src/
│   ├── common/              Shared C++ utilities
│   │   ├── framing.h        Frame protocol (header-only)
│   │   ├── socket_utils.h   TCP abstraction
│   │   └── socket_utils.cpp
│   ├── injector/            Standalone injection CLI
│   │   ├── main.cpp         Entry point, argument parsing, --list-deps, exit codes 1-6
│   │   ├── injector.h       Public API declarations
│   │   ├── injector_win.cpp Win32 implementation (CreateRemoteThread, PE
│   │   │                    import parser, dependency-closure preload)
│   │   ├── injector_di.cpp  DI variants (IProcessOps-driven, fully testable)
│   │   └── os_ops.h         IProcessOps / MockProcessOps / Win32ProcessOps
│   └── library/             Injected DLL
│       ├── entry_win.cpp    DllMain / Windows entry
│       ├── api.h            InitParams handshake layout (1024 bytes)
│       ├── compat_qt.h      Qt5/Qt6 compatibility macros
│       ├── core/            UI scanner, event injector, screenshot, element map
│       ├── rpc/             TCP RPC server (JSON-RPC handler)
│       └── selector/        Element query engine
│
├── tests/
│   ├── unit_server/         Python unit tests (254)
│   ├── unit_injector/       C++ unit + E2E tests (326)
│   ├── unit_library/        C++ library component tests (43)
│   ├── verify_preload.py    E2E: dependency preload scenarios A/B/C
│   └── test-apps/           Minimal Qt test applications
│
└── CMakeLists.txt

License

Dual-licensed: MIT AND the
Anti 996 License — free to use, modify, distribute,
and integrate into commercial projects, with attribution, provided both
licenses are satisfied. See LICENSE for the summary.

Author

Developed and maintained by TieFeiyu.