Accio brings macOS interaction into normal Python, so an agent can use the GUI only where the GUI is needed and combine it with shell commands, app CLIs, APIs, and files in the same task.

• Hybrid by default. Desktop actions and ordinary code share one execution block, including imports, conditions, loops, parsing, and local tools.
• Fewer model round trips. A block can complete dependent steps, branch on returned state, and recover locally instead of asking the model to plan every click separately.
• Observable and verifiable. GUI actions return refreshed accessibility state and screenshot artifacts; typed Python signatures catch invalid calls before they reach the desktop.
• Reusable after success. Code written for one task can be kept as a function or workflow instead of remaining an unstructured action trace.

Quick start

Accio currently supports macOS 14 or later. You will need Swift 6,
Python 3.10 or later, Accessibility permission, and Screen Recording permission.

Version 0.0.1 is a source-only release. Build and install it locally using the
scripts below; the project does not currently distribute a prebuilt,
notarized application.

1. Install

From the repository root, run:

git clone https://github.com/Accio-Work-Lab/accio-cu.git
cd accio-cu
./scripts/install-macos.sh --verify --install-skill

This installs the app-bundled accio-computer-use command, its Python runtime, and the Accio skill for Codex. To install the skill for Claude Code instead:

./scripts/install-macos.sh --install-skill claude

The installer uses ad-hoc signing by default. A rebuilt ad-hoc app may have a
new macOS permission identity and require Accessibility and Screen Recording
to be granted again. To preserve the TCC identity across builds, install every
build with the same certificate:

ACCIO_CODESIGN_IDENTITY="Developer ID Application: Example (TEAMID)" \
  ./scripts/install-macos.sh --verify --install-skill

To remove the app, daemon, CLI link, and Accio's scoped TCC grants:

./scripts/install-macos.sh --uninstall

2. Grant permissions

accio-computer-use setup

Enable Accessibility and Screen Recording for Accio Computer Use in System Settings → Privacy & Security.

3. Start the daemon

./scripts/install-daemon.sh install
./scripts/install-daemon.sh status

Restart your agent, then give it a desktop task:

Open Finder, create a folder named Research Notes in Documents,
move the three PDF files into it, and verify the final folder contents.

The installed skill handles observation, interaction, and final verification.

One block, many tools

Pipe Python into accio-computer-use; the desktop functions are already loaded:

accio-computer-use <<'PY'
from pathlib import Path

def create_note(source_file):
    body = Path(source_file).expanduser().read_text()

    click(app="Notes", element_text="New Note")
    type_text(app="Notes", text=body)

    final = get_app_state(app="Notes")
    return {
        "saved": body[:40] in final.text,
        "screenshots": final.screenshot_paths,
    }

emit(create_note("~/Documents/research-notes.md"))
PY

This block reads a file, applies Python logic, and operates a native app. The same block could call an app CLI or API before using the GUI. A model can write it while solving the task, then keep the working code for the next run.

For execution limits or a custom daemon socket, use the explicit code subcommand:

accio-computer-use code --timeout 60 --max-calls 30 < workflow.py

MCP

The same coding runtime is available as one MCP tool named execute:

{
  "mcpServers": {
    "accio-computer-use": {
      "command": "accio-computer-use",
      "args": ["code", "mcp", "--timeout", "60", "--max-calls", "30"]
    }
  }
}

The tool accepts a Python program and returns its output, desktop calls, and screenshot artifacts. Limits are fixed when the MCP server starts.

This hybrid MCP is started with code mcp. The separate accio-computer-use mcp command is the lower-level interface that exposes individual desktop actions instead of the single execute tool.

Coding MCP is intentionally opt-in. It executes normal Python with the current
user's permissions and must be enabled only for trusted clients and code.

MacOSWorld results

Completed runs and planned comparisons:

Across all 202 non-safety MacOSWorld tasks, Accio hybrid improved on the legacy run by 6.43 points, with 43.2% fewer recorded tokens and 53.5% less recorded time. Only a score of 100 counts as success; because the runs used different model versions, this is an end-to-end comparison rather than a controlled harness ablation.

Current limits

• macOS 14 or later only
• Accessibility and Screen Recording permissions are required
• Some Electron and non-native apps may briefly become frontmost during an input fallback
• Some background input routes use private SkyLight APIs and may need updates after macOS changes
• coding mode runs normal Python with the caller's filesystem, environment, network, and user-session access; it is not a sandbox

Use an isolated VM or dedicated macOS session for untrusted agents and large rollout workloads.

Security

Accio is a local, high-privilege automation tool:

• the daemon socket uses the current macOS user as its trust boundary;
• screenshots, accessibility state, logs, and traces may contain private data;
• coding mode is unsandboxed and can access the current user's files,
  environment, network, and processes;
• SkyLight integration uses unsupported Apple private APIs and is not intended
  for Mac App Store distribution.

Do not expose the daemon or MCP transports to a network. Use a dedicated macOS
account or VM for untrusted agents. See SECURITY.md for reporting
and deployment guidance.

Development

swift build
swift test
swift build -c release --product AccioComputerUse
python3 -m unittest discover -s coding/tests -p 'test_*.py'
bash -n scripts/install-macos.sh
bash -n scripts/install-daemon.sh

See CONTRIBUTING.md for contribution and platform-testing
requirements.

Authors

Tianyuan Yang^*, Yu Gu^*, Lanbo Lin, Yuanwu Xu, Li Cai,
and Sicong Xie

^* Equal contribution.

Citation

If you use Accio Computer Use in your work, please cite this repository:

@misc{Yang2026Accio,
    author={Tianyuan Yang and Yu Gu and Lanbo Lin and Yuanwu Xu and Li Cai and Sicong Xie},
    title={Accio Computer Use},
    note={GitHub repository},
    howpublished={\url{https://github.com/Accio-Work-Lab/accio-cu}},
    year={2026}
}

License

Accio Computer Use is licensed under the
Apache License 2.0. Attribution information is in NOTICE.