CesiumJS AI Sample App

A ready-to-run starter pairing a CesiumJS 3D globe with an LLM-powered chat interface. The LLM drives the globe through structured tool calls (e.g. "fly to Paris") while the LLM API key never reaches the browser — all inference runs behind a Node.js API server.

Getting Started  ·  Architecture  ·  Packages  ·  Viewer Tools Tutorial

Get Started

You need an LLM API key from OpenAI, Anthropic, or Google Generative AI to enable chat. A free Cesium Ion token is optional but improves terrain and imagery.

npx degit CesiumGS/cesiumjs-ai-starter-app cesiumjs-ai-starter-app
cd cesiumjs-ai-starter-app
cp .env.example .env

Open .env and set AI_PROVIDER with its matching API key (e.g. OPENAI_API_KEY). Optionally set VITE_CESIUM_ION_ACCESS_TOKEN.

Option A — Docker (recommended)

Requires Docker Desktop only — no local Node.js needed.

docker compose up --build --wait

Open http://localhost:8080 once both containers report healthy.

Option B — Local dev (hot reload)

Requires Node.js ≥ 20 and npm ≥ 9.

npm install
npm run dev

• Globe + HMR: http://localhost:5173
• Chat API: http://localhost:3001

Try It Out

Type a place into the chat panel — e.g. fly to Paris — and the camera flies there. Any city, landmark, or address works.

For requests that don't fit a single fly-to, try something like drop a pin at the Eiffel Tower — this routes through executeCesiumCode instead, which is gated behind AI SDK's native tool-approval mechanism (toolApproval configured in backend/src/app.ts — see backend/src/tools/execute-cesium-code-tool.ts): the chat panel shows you the assistant's raw intent for the call and waits for you to approve or reject it (a human-in-the-loop checkpoint — see frontend/src/components/ChatPanel.tsx's onApprovalRequired handler) before any code is generated. After approval, the backend generates and statically verifies a CesiumJS snippet; the frontend then executes it in a fresh QuickJS-WASM sandbox with memory/deadline limits and a guarded bridge to the live Viewer.

If no provider key is configured, the globe still runs as a plain viewer with a "AI is not configured. Add a supported provider API key to your .env file." banner.

Architecture

Viewer tools (camera, entities) are streamed via Server-Sent Events to the browser and executed against the live Viewer. The LLM API key never reaches the browser — all inference happens behind the Node.js API. See Architecture for the full component diagram, request sequence, and deployment topology. The workspace packages provide the reusable pieces:

• @cesium-ai/server — an Express router that mounts the AI SDK chat key-layer (/api/chat). It accepts a tool registry and a resolved language model and runs the streamText agent loop server-side, so the LLM API key never reaches the browser. The host app owns provider selection.
• @cesium-ai/tools-schemas — Zod-schemed CesiumJS viewer tool definitions (flyTo, …). Schemas only, no execute, and scoped strictly to tools that run directly against a live Viewer.
• @cesium-ai/tools — default, ready-to-use client-side executors for every tool in @cesium-ai/tools-schemas's catalogue (flyTo, camera, entity, animation, and imagery tools) — the browser-side "other half" of that schema-only package, so a host app doesn't have to hand-write an executor for every tool before it can turn one on. createCesiumToolExecutors({ ... }) lets a host override or extend any individual tool (e.g. this app's own flyTo, which validates against an extended shape — see below) without forking the rest. See packages/tools/README.md.
• @cesium-ai/codegen-cesium — backend-only pipeline that turns executeCesiumCode's natural-language intent into statically-verified CesiumJS code (skills-grounded generation + an AST verifier), and also owns executeCesiumCode's tool definition itself (schema-only, no execute) — that tool can't run directly against a Viewer like flyTo does, so it lives here rather than in tools-schemas. Parse-only — it never executes generated code itself.
• @cesium-ai/codegen-sandbox — frontend-only QuickJS-WASM execution sandbox used to run verified generated CesiumJS snippets under strict memory/time limits and a guarded bridge to the live Viewer.
• @cesium-ai/mcp-tools — optional, server-only Model Context Protocol client bridge. Connects to MCP servers (SSE/HTTP — stdio is deliberately unsupported), namespaces + allowlist-filters their tools, and merges them into an AI SDK ToolSet a host app spreads alongside createCesiumTools() — this is the "MCP-backed tool group" the split-execution diagram above refers to. Entirely opt-in through an mcp.config.json file, with no MCP client created when the file is absent. See packages/mcp-tools/README.md for the full security model and API.
• @cesium-ai/chat-element — reusable React chat UI package consumed by the frontend app, including tool event rendering and inline approval UI used for protected tool calls.

This app builds its own executable executeCesiumCode tool on top of the library's schema (backend/src/tools/execute-cesium-code-tool.ts, wrapping @cesium-ai/codegen-cesium), the same "app extends the shared schema" pattern flyTo uses via backend/src/tools/flyto-tool.ts. Because executeCesiumCode is a "Code Mode" tool — the model's output is arbitrary generated code, not bounded typed args like flyTo's lat/lon/altitude — it needs a materially different security posture than flyTo. The backend's AST verification (see packages/codegen-cesium/README.md) is defense-in-depth only, not a substitute for runtime isolation; the frontend independently executes verified snippets through @cesium-ai/codegen-sandbox, a fresh QuickJS-WASM interpreter with memory/deadline limits and a guarded host bridge. See packages/tools-schemas/README.md and packages/codegen-cesium/README.md for the full generation/verification pipeline.

See Architecture for more detail.

Working with Cesium Tools

Enable or disable a tool: edit ENABLED_CESIUM_TOOLS in shared/src/enabled-tools.ts. Both the backend registry and frontend executor map derive from this array — a typo fails to build, and enabling a tool without a client-side executor also fails to compile.

Update a tool's schema: structural args rules live in flyToInputShape (both tiers derive from it); model-facing descriptions live in flyTo.ts backend-only and are never bundled into the client.

See the Cesium Viewer Tools Tutorial for the full step-by-step walkthrough.

Enabling MCP tools

MCP tools are a separate, opt-in tool group — see @cesium-ai/mcp-tools for the full API and security model. Nothing below is required for the app to run; without an mcp.config.json file, no MCP client is ever created.

List servers in a dedicated mcp.config.json file at the repo root (see mcp.config.json.example). It is a plain JSON array, easier to read and edit than an inline environment value, and keeps server configuration out of shell history:

[
  {
    "name": "docs",
    "transport": { "type": "http", "url": "https://example.com/mcp" },
    "allowedTools": ["search"]
  },
  { "name": "ion", "transport": { "type": "http", "url": "http://localhost:3000/mcp/" } }
]

There's no manual "does this need OAuth" flag to set. backend/src/index.ts attempts every configured server the same way at startup (createMcpTools):

• Connects successfully (e.g. the docs server above, using a static header/API key or no auth at all) — shared by every visitor from then on. Tools are namespaced mcp__<name>__<toolName> and merged into the same registry flyTo/executeCesiumCode live in (backend/src/app.ts) — every MCP tool is approval-gated by default (toolApproval: "user-approval"), the same human-in-the-loop checkpoint executeCesiumCode uses, since MCP tools run arbitrary third-party server code this app doesn't control.
• Fails with a 401 (e.g. the ion server above) — auto-detected as needing per-user authentication, and automatically offered through the chat panel's interactive Connect flow instead: credentials and MCP clients are scoped to the browser session, kept in backend memory only, and discarded on disconnect or restart. See the mcp-tools README's session-scoped OAuth section.
• Fails any other way (network unreachable, bad URL, etc.) — recorded as a genuine connection failure (surfaced on GET /health as mcpServers) and never prevents the rest of the app from starting.

The full resolved tool registry for a given request — including any connected MCP tools, which aren't knowable statically at build time — can be inspected via GET /api/tools (rate-limited the same as /api/chat); the frontend's ChatPanel shows this list in a small "Tools (N)" disclosure so it's visible which tools (built-in and MCP) are actually available in a running instance.

If a connected server's tool declares an "MCP Apps" ui:// widget resource (e.g. an interactive asset-import launcher), GET /api/tools reports it via that tool's mcpApp field and the chat panel renders it inline in a sandboxed iframe (packages/chat-element/src/McpAppWidget.tsx) — including a bridge that lets the widget call tools back on its own server, gated behind an explicit inline Approve/Reject prompt. See the mcp-tools README's MCP Apps widgets section for the full architecture.

Environment Variables

See .env.example for the complete list, including AI_BASE_URL and telemetry settings for OTLP-compatible providers.

All VITE_* vars are read from this single repo-root .env for both npm run dev (via frontend/vite.config.ts's envDir) and docker compose up (via compose.yaml's build-arg substitution) — there is no separate frontend/.env to keep in sync.

Copy .env.example to .env and fill in your values. The .env file is git-ignored.

Scripts

Project Structure

cesiumjs-ai-starter-app/
├── frontend/              # Vite + React SPA (CesiumJS globe + chat panel)
├── backend/               # Thin Express host (provider selection, tool registry, API key)
├── packages/
│   ├── server/            # @cesium-ai/server — chat router + agent loop
│   ├── tools-schemas/     # @cesium-ai/tools-schemas — viewer tool schemas
│   ├── tools/             # @cesium-ai/tools — default client-side tool executors
│   ├── codegen-cesium/    # @cesium-ai/codegen-cesium — codegen pipeline + executeCesiumCode tool
│   ├── codegen-sandbox/   # @cesium-ai/codegen-sandbox — frontend sandbox for generated code
│   ├── mcp-tools/         # @cesium-ai/mcp-tools — optional MCP client bridge
│   ├── chat-element/      # @cesium-ai/chat-element — reusable chat panel component
├── shared/                # @cesium-ai/sample-config — enabled tools + flyTo args contract
├── compose.yaml           # Docker Compose — frontend + backend on internal network
└── .env.example           # Environment variable template

Code Style

Formatted with Prettier (printWidth: 100, otherwise defaults). Run before committing — CI fails on unformatted files.

npm run format        # format in place
npm run format:check  # verify (CI)

Contributing & License

Contributions welcome — see CONTRIBUTING.md and CODE_OF_CONDUCT.md.

Apache 2.0 — see LICENSE.