pilot — Your AI Agent, Inside Your Real Browser

Your AI agent controls a tab in your real Chrome — already logged in, no bots blocked, no CAPTCHAs.

Other browser tools launch a separate headless browser. Your agent starts anonymous, gets blocked by Cloudflare, can't access anything behind login.

pilot takes a different approach: it controls a tab in the browser you're already using. Your agent sees what you see — logged into GitHub, Linear, Notion, your internal tools. No cookie hacks. No re-authentication. No bot detection.

---

Quick Start

1. Install pilot

npx pilot-mcp
npx playwright install chromium

Add to .mcp.json (Claude Code) or MCP settings (Cursor):

{
  "mcpServers": {
    "pilot": {
      "command": "npx",
      "args": ["-y", "pilot-mcp"]
    }
  }
}

2. Install the Chrome extension

npx pilot-mcp --install-extension

This opens Chrome's extensions page and shows the folder path. Click Load unpacked → paste the path. You'll see the ✈️ Pilot icon — badge shows ON when connected.

3. Use it

Tell your agent:

"Go to my GitHub notifications and summarize them"

The agent navigates in a real Chrome tab — already logged in as you. No setup. No cookies. No Cloudflare blocks.

---

Two Modes

Extension Mode — your real browser

The Pilot Chrome extension connects to the MCP server via WebSocket. Your agent gets its own tab in your real browser — with all your sessions, cookies, and logged-in state already there.

AI Agent → MCP (stdio) → pilot → WebSocket → Chrome Extension → Your Browser Tab

• No Cloudflare blocks (real browser fingerprint)
• Already authenticated everywhere
• Multiple agents get separate tabs (multiplexed)
• You can watch the agent work in real-time

This is how pilot is meant to be used.

Headed Mode — visible Chromium

When the extension isn't connected, pilot opens a visible Chromium window. You can see everything the agent does and intervene when needed.

Import cookies from your real browser to authenticate:

pilot_import_cookies({ browser: "chrome", domains: [".github.com", ".linear.app"] })

Supports Chrome, Arc, Brave, Edge, Comet via macOS Keychain / Linux libsecret.

When the agent hits a CAPTCHA or bot wall, it hands control to you:

1. pilot_handoff — pauses automation, you solve the challenge
2. pilot_resume — agent continues where it left off

---

Lean Snapshots

Large page snapshots eat context windows. pilot is opinionated about keeping things small:

• Navigate returns a ~2K char preview, not a 50K+ page dump
• Snapshot supports max_elements, interactive_only, lean, structure_only
• Snapshot diff shows only what changed — no redundant re-reads

Other tools:   navigate(58K) → navigate(58K) → answer        = 116K chars
pilot:         navigate(2K)  → navigate(2K)  → snapshot(9K)  =  13K chars

Less context = faster inference, cheaper API calls, fewer failures.

---

pilot vs @playwright/mcp

Both are solid tools. Here's what's actually different:

	pilot	@playwright/mcp
Real browser control	Extension controls a tab in your Chrome	Extension for session reuse (no DOM control)
Bot detection	Not an issue (real browser) + handoff/resume	❌ blocked by Cloudflare
Cookie import	Decrypt from Chrome, Arc, Brave, Edge, Comet	❌ (manual --storage-state JSON)
Default snapshot size	~2K on navigate, ~9K full snapshot	~50-60K on navigate
Snapshot diffing	pilot_snapshot_diff	❌
Token control	max_elements, interactive_only, lean, structure_only	--snapshot-mode (incremental/full/none)
Iframe support	pilot_frames, pilot_frame_select, pilot_frame_reset	❌
Ad blocking	pilot_block with ads preset	--blocked-origins (manual)
Tool profiles	core (9) / standard (30) / full (61)	Capability groups via --caps
Transport	stdio	stdio, HTTP, SSE
Persistent sessions	pilot_auth + cookie import	--user-data-dir, --storage-state
Network interception	pilot_intercept	browser_route
Assertions	pilot_assert	Verify tools via --caps=testing

Use pilot when: You need your agent to work on authenticated sites, you want lean context, or you're tired of Cloudflare blocks.

Use @playwright/mcp when: You need HTTP/SSE transport, Windows auth support, or you prefer Microsoft's ecosystem.

---

Tool Profiles

61 tools is too many for most LLMs — research shows degradation past ~30. Load only what you need:

Profile	Tools	Use case
core	9	Simple automation — navigate, snapshot, click, fill, type, press_key, wait, screenshot
standard	30	Common workflows — core + tabs, scroll, hover, drag, iframes, auth, block, find
full	61	Everything, including network mocking, assertions, clipboard, geolocation

{
  "mcpServers": {
    "pilot": {
      "command": "npx",
      "args": ["-y", "pilot-mcp"],
      "env": { "PILOT_PROFILE": "standard" }
    }
  }
}

Default is standard (30 tools).

---

All Tools (61)

Navigation

Tool	Description
pilot_get	Navigate and return full readable content + interactive elements in one call
pilot_navigate	Navigate to a URL. Returns content preview + interactive elements (~2K chars)
pilot_back	Go back in browser history
pilot_forward	Go forward in browser history
pilot_reload	Reload the current page

Snapshots

Tool	Description
pilot_snapshot	Accessibility tree with @eN refs. Supports max_elements, structure_only, interactive_only, lean, compact, depth
pilot_snapshot_diff	Unified diff showing what changed since last snapshot
pilot_find	Find element by visible text, label, or role — returns a ref without a full snapshot
pilot_annotated_screenshot	Screenshot with red boxes at each @ref position

Interaction

Tool	Description
pilot_click	Click by @ref or CSS selector
pilot_hover	Hover over an element
pilot_fill	Clear and fill an input/textarea
pilot_select_option	Select a dropdown option
pilot_type	Type text character by character
pilot_press_key	Press keyboard keys
pilot_drag	Drag from one element to another
pilot_scroll	Scroll element or page
pilot_wait	Wait for element, network idle, or page load
pilot_file_upload	Upload files to a file input

Iframes

Tool	Description
pilot_frames	List all iframes
pilot_frame_select	Switch context into an iframe
pilot_frame_reset	Switch back to main frame

Page Inspection

Tool	Description
pilot_page_text	Clean text extraction
pilot_page_html	Get innerHTML of element or full page
pilot_page_links	All links as text + href pairs
pilot_page_forms	All form fields as structured JSON
pilot_page_attrs	All attributes of an element
pilot_page_css	Computed CSS property value
pilot_element_state	Check visible/hidden/enabled/disabled/checked/focused
pilot_page_diff	Text diff between two URLs

Debugging

Tool	Description
pilot_console	Console messages from circular buffer
pilot_network	Network requests from circular buffer
pilot_dialog	Captured alert/confirm/prompt messages
pilot_evaluate	Run JavaScript on the page
pilot_cookies	Get all cookies as JSON
pilot_storage	Get localStorage/sessionStorage
pilot_perf	Page load performance timings

Visual

Tool	Description
pilot_screenshot	Screenshot of page or element
pilot_pdf	Save page as PDF
pilot_responsive	Screenshots at mobile, tablet, desktop

Tabs

Tool	Description
pilot_tabs	List open tabs
pilot_tab_new	Open a new tab
pilot_tab_close	Close a tab
pilot_tab_select	Switch to a tab

Session & Auth

Tool	Description
pilot_import_cookies	Import cookies from Chrome, Arc, Brave, Edge, Comet via Keychain decryption
pilot_auth	Save/load/clear full session state (cookies + localStorage + sessionStorage)
pilot_set_cookie	Set a cookie manually
pilot_set_header	Set custom request headers
pilot_set_useragent	Set user agent string
pilot_handle_dialog	Configure dialog auto-accept/dismiss
pilot_resize	Set viewport size
pilot_block	Block requests by URL pattern or ads preset
pilot_geolocation	Set fake GPS coordinates
pilot_cdp	Connect to a real Chrome instance via CDP
pilot_extension_status	Check Chrome extension connection status
pilot_handoff	Open headed Chrome for manual interaction (CAPTCHA, auth)
pilot_resume	Resume automation after handoff
pilot_close	Close browser and clean up

Automation (full profile)

Tool	Description
pilot_intercept	Intercept requests and return custom responses
pilot_assert	Assert URL, text, element state, or value
pilot_clipboard	Read or write clipboard content

---

Extension Architecture

The Pilot extension uses a broker/client model — multiple AI sessions share one extension, each getting its own tab:

Claude Code Session A ──┐
                        ├→ pilot broker (ws://127.0.0.1:3131) → Chrome Extension → Tab 1
Claude Code Session B ──┘                                                       → Tab 2

Each session's tab is color-grouped in Chrome so you can see which tab belongs to which agent.

---

Requirements

• Node.js >= 18
• Chrome + Pilot extension (recommended)
• macOS or Linux (for cookie import in headed mode)
• Chromium: npx playwright install chromium (for headed mode)

Security

Variable	Default	Description
PILOT_PROFILE	standard	Tool set: core (9), standard (30), or full (61)
PILOT_OUTPUT_DIR	System temp	Restricts where screenshots/PDFs can be written

• Extension communicates over localhost WebSocket only (127.0.0.1)
• Output path validation prevents writing outside PILOT_OUTPUT_DIR
• Path traversal protection on all file operations
• Expression size limit (50KB) on pilot_evaluate

Development

npm test   # unit tests via vitest

---

Credits

The core browser automation architecture — ref-based element selection, snapshot diffing, cursor-interactive scanning, annotated screenshots, circular buffers, and AI-friendly error translation — is ported from gstack by Garry Tan.

Built on Playwright by Microsoft and the Model Context Protocol SDK by Anthropic.

---

If pilot is useful to you, star the repo — it helps others find it.