OpenCanvas

Stop reading AI replies. Watch them assemble on an infinite canvas.

Your LLM doesn't print markdown. It places typed widgets on a tldraw canvas: charts, diagrams, code, kanbans, tables, custom HTML. It learns your patterns and registers reusable widget templates as it goes. Any model with function calling. Bring your own key.

BYO model · MCP-native · Self-improving · Local-first · MIT

▶︎ Try the live demo  ·  ⭐ Star on GitHub  ·  📡 API reference  ·  🏗 Architecture

Install · Why · Models · Widgets · REST API · Security

Why OpenCanvas

Chat apps give you a scrolling wall of text. OpenCanvas gives the model a canvas and a toolbox.

Ask for a sales breakdown and you get a Vega-Lite chart next to a kanban of follow-ups next to the SQL that produced it. Drag them around. Pin the ones you keep. Every widget is a typed contract, so the model can't hand you broken output, and every conversation lands in a searchable local knowledge base.

Under the hood, that translates to:

Install

git clone https://github.com/ashark-ai-05/opencanvas.git
cd opencanvas
pnpm install
cp .env.example .env       # at least one provider key
pnpm electron:dev          # backend + Vite + Electron, one command

Headless without Electron: pnpm dev, then open http://127.0.0.1:3458.

Packaged installers (.dmg / .exe / .AppImage) ship on the Releases tab. Want a public demo of your own? docs/deploy-railway.md gets you there in about ten minutes.

Pick any LLM

Every provider gets the same agent: same tools, same widget surface, same UX. The model is a config switch.

Tool calling runs on the Vercel AI SDK, so any provider with function-calling support behaves the same. Embedders are pluggable too: bundled ONNX (offline), OpenAI, Voyage, or Ollama. Architecture notes live in docs/plans/unified-agent.md.

// ~/.opencanvas/config.json
{
  "activeProfile": "local",
  "profiles": [{
    "name": "local",
    "llm":   { "provider": "ollama", "model": "llama3.1:8b" },
    "embed": { "provider": "ollama", "model": "nomic-embed-text" },
    "sources": []
  }]
}

The widget catalog

15 kinds. Markdown · code · tables · timelines · file trees · kanban · tasks · sticky notes · composite cards · charts (Vega-Lite) · calendars · clock / timer / stopwatch / pomodoro · web embeds (sandboxed iframes) · plugin iframes · generic fallback.

Every widget is a typed contract: a Zod schema for props plus a React component. Add a new kind without retraining anything.

Runtime plugins. The agent can call register_widget_kind mid-conversation to declare a brand-new widget on the fly. A Python REPL (Pyodide) and a JS REPL ship as examples. Plugin iframes run with sandbox="allow-scripts" and no allow-same-origin: null origin, no access to the parent DOM.

MCP-native

Add any MCP server under profiles[].sources. Works with every provider.

{
  "sources": [{
    "id": "dev-fs",
    "transport": "stdio",
    "command": "npx",
    "args": ["-y", "@modelcontextprotocol/server-filesystem", "/Users/me/Development"]
  }]
}

Verify with pnpm cli --probe-sources. The agent calls them as mcp__dev-fs__<tool>.

Drive it from any process

Cron jobs, file watchers, shell scripts: anything on your machine can render widgets. The browser holds an SSE connection to /v1/canvas/events; external POSTs push directives onto the per-conversation event bus.

# Place a Vega-Lite chart from your terminal
curl -X POST http://127.0.0.1:3457/v1/canvas/widgets \
  -H 'content-type: application/json' \
  -H "x-opencanvas-token: $(cat ~/.opencanvas/auth-token)" \
  -d '{"kind":"chart","role":"primary","payload":{"title":"Sales","spec":{...}}}'

REST reference. Browse the live API docs or read the OpenAPI 3.1 spec. A running backend serves the same UI at http://127.0.0.1:3457/docs.

Auth. A 256-bit token is generated on first run and stored at ~/.opencanvas/auth-token (mode 0600). Required on every state-mutating route. Details in SECURITY.md.

Self-improving knowledge base

Every conversation auto-indexes into the same SQLite + sqlite-vec store as your docs and code. Old turns become searchable through search_kb. Retrieval is hybrid BM25 + vector. Re-indexing is idempotent: unchanged content costs zero LLM calls.

pnpm cli --index ./docs            # markdown / text → chunks + embeddings
pnpm cli --index-code ./src        # tree-sitter chunks + symbols
pnpm cli --search "<query>"        # hybrid search

The agent also learns you. Widget kinds you keep and pin score higher next time; kinds you dismiss get placed less. Per conversation, stored locally, no remote tracking.

Architecture

┌────────────────────┐                ┌─────────────────────────┐
│  Vite + React +    │   /v1/chat     │  Hono backend           │
│  tldraw            │ ─────────────→ │                         │
│  • floating chat   │                │  provider adapters      │
│  • canvas          │                │  12 in-process tools    │
│  • palette ⌘K      │ ←── SSE ────── │  + external MCP servers │
│  • history         │  /v1/canvas/   │  + plugin registry      │
└────────────────────┘    events      │       │                 │
                                      │       ▼                 │
                                      │  SQLite + sqlite-vec    │
                                      └─────────────────────────┘

tldraw v3 with one shape util per widget kind. Zustand for app state. AI SDK for chat streaming. Hono + better-sqlite3 + sqlite-vec on the backend. Electron wrapper for the desktop build.

Development

pnpm test                                          # backend (vitest, Node)
pnpm vitest run --config app/vite.config.ts        # frontend (vitest + jsdom)
pnpm typecheck                                     # tsc --noEmit
pnpm app:build                                     # production bundle
pnpm dist                                          # electron-builder installer

610 tests: 396 backend, 214 frontend. CI runs both suites plus a production dependency audit on every push.

Status

Experimental. Solo project, MIT. No telemetry, no remote services unless you point it at one.

Issues and PRs welcome. See CONTRIBUTING.md. Security reports go through SECURITY.md.