Skillget

iphone-agent-harness

agent
Security Audit
Fail
Health Warn
  • License — License: MIT
  • Description — Repository has a description
  • Active repo — Last push 0 days ago
  • Low visibility — Only 5 GitHub stars
Code Fail
  • network request — Outbound network request in apps/web/api/openai-relay-worker.js
  • exec() — Shell command execution in apps/web/api/worker.js
  • network request — Outbound network request in apps/web/api/worker.js
  • crypto private key — Private key handling in apps/web/community/identity.js
Permissions Pass
  • Permissions — No dangerous permissions requested

No AI report is available for this listing yet.

SUMMARY

Automate your real iPhone from Safari with a $2 ESP32. No Mac, no APP, and no jailbreak. Local conversations, your choice of AI, and community skills.

README.md

iPhone Agent Harness

iPhone Agent Harness

Let AI automate your iPhone.

English · 简体中文 · 繁體中文 · 日本語 · Русский · Deutsch · Français · Español

CI MIT License iOS / iPadOS 27+ ESP32 C2, C3, C5, C6, S3, Classic

📱 Open Harness · ⚡ Browser installer · 📖 Full guide · 🧩 Skill community · 🗺️ Architecture

Give your iPhone a task in plain language: inspect a screen, enter text, adjust a control or follow a short workflow across apps. A small powered ESP32 bridge supplies Bluetooth input while Safari runs the conversation and screen capture.

  • Install in your browser. Choose your ESP32 and flash its firmware with desktop Chrome or Edge.
  • Keep your phone wireless. Leave the powered bridge on a shelf or in a room corner within Bluetooth range.
  • Use your normal Wi-Fi. The phone and bridge share your home or office network. Open Safari to start a task.

https://github.com/user-attachments/assets/d72669a6-e150-40d7-ba2f-1d9586d9c2ea

An agent sets an alarm on a real iPhone.

⚙️ How it works

Safari captures the phone screen and sends selected context to your chosen AI. The Harness validates an action plan, sends it directly to the ESP32 over local WebRTC, and the bridge delivers Bluetooth mouse/keyboard input. The next screen observation closes the loop.

flowchart LR
  subgraph Phone["iPhone / iPad"]
    Harness["Safari · Harness"]
    Apps["iPhone / iPad apps"]
    Storage[("Local browser storage<br/>Sessions · images · keys")]
    Apps -->|Screen sharing| Harness
    Harness <--> Storage
    Shortcut["Apple Shortcuts<br/>iOS clipboard"]
  end
  Harness <-->|API / MCP| AI["Your chosen AI"]
  Harness <-->|ChatGPT subscription| Relay["Stateless relay"]
  Relay <--> AI
  Harness <-->|WebRTC · local Wi-Fi| Bridge["ESP32 bridge"]
  Bridge -->|Bluetooth keyboard / mouse| Apps
  Bridge -->|Local HTTP · text| Shortcut
  Shortcut -->|Copy confirmed| Harness

Architecture details

🔌 Choose a board

A basic ESP32-C3 board costs roughly US$2. Pick your chip in the browser installer, plug it into your computer with a USB data cable, and flash.

Chip Supported models
ESP32-C2 2 MB minimum · 26/40 MHz crystal detected automatically
ESP32-C3 All models*
ESP32-C5 All models*
ESP32-C6 All models*
ESP32-S3 All models*
ESP32 (classic) All models*

*The current images for C3, C5, C6, S3 and classic ESP32 use standard SPI flash with at least 4 MB capacity.

🚀 Set up and try it

The complete guide covers every step. These links take you to the relevant section:

  1. Enable accessibility

  2. Flash firmware

  3. Join Wi-Fi

  4. Pair Bluetooth

  5. Open Harness

  6. Add AI access

  7. Connect and test the bridge

  8. Install the clipboard Shortcut

  9. Install a community skill

  10. Create your own skill

  11. Publish a skill

💬 Conversations, plans and skills

Ask a question, work through a plan or give the agent a task in the same conversation. Conversations support rename, archive, search and file export. Scroll back through the history while a task runs, then jump to the latest reply whenever you want.

Skills teach the agent how to handle common phone tasks and improve its taps, typing and gestures. Add skills from the community or ask Harness to help write your own. You can also connect MCP tools in Settings.

Actual Harness screens with sample content; the conversation below is an example.

Your conversationsExample conversationSkills
Your conversations Example conversation Skills

One bridge supports two independent Harness connections. When using two phones, select the matching phone on each page and run the optional input test on each to confirm the pairing.

📋 Long text, in any language

The bridge handles everyday typing and shortcuts as a Bluetooth keyboard. For long or multilingual text, iPhone Agent Clipboard uses Apple Shortcuts to copy the text, then the agent pastes it into the chosen field.

The browser sends the text to the ESP32 in small chunks over local WebRTC. The bridge streams those chunks in one HTTP response to the Shortcut, which puts the complete text on the iPhone clipboard.

Set up clipboard copy

🔒 Data and privacy

Sessions, screenshots and credentials are saved in your browser; ordinary APIs connect directly to your provider. ChatGPT subscriptions use the project’s Cloudflare relay or your own stateless relay because of browser CORS, forwarding selected context, images, output and a short-lived access token. The relay application does not persist this traffic or log bodies or credentials.

🤝 Contribute

Ideas, bug reports and improvements are welcome: open an issue or send a pull request. Share your custom skills in the skill community.

MIT License · Third-party notices

💛 Acknowledgements

Thanks to onehumanbeing/esp32-iphone-agent for inspiration.

Reviews (0)

No results found