OpenGenie

The Databricks coding agent, on the model you choose.

Website and documentation: https://raphaelmansuy.github.io/opengenie/ — the tour, the install
guide, the evidence page and the design notes. The source is on
GitHub; the current release is
v1.0.0.

Genie Code is the best coding agent for data engineering because it can see your workspace: Unity
Catalog, SQL warehouses, notebooks, jobs. What it does not let you do is choose the model, or control
what the work costs. OpenGenie is an open-source Databricks App that gives a coding agent the same kind
of workspace context and hands those two decisions back to you.

The app deployed in a Databricks workspace. The notebook is the centre of the screen; the assistant
sits beside it and knows which notebook the conversation is about.

Why it exists

Genie Code got the important part right: the agent works inside Databricks, so its answers come from
your data instead of from a guess. Two things come with it that a team cannot change: the model is
chosen for you, and the bill is metered by a product you do not configure.

Raphael Mansuy built OpenGenie to keep the context and
return those decisions to the team. Before building it, he measured whether the model matters. Same
agent, same tools, same rules, on eight development tasks scored in a live workspace. The score is
the scoring points earned out of the points available across those tasks:

One run each. The caveats and the command that regenerates these numbers are on the
evidence page. The model was the largest lever
we could measure, and it is the one a Genie Code user cannot pull.

What the agent can reach

Each conversation gets its own MCP tool server. In a deployment it calls
Databricks as the signed-in user, so Unity Catalog permissions still decide what the agent sees. Run
locally, it acts as the owner of the token you give it.

That is the 28-tool core profile the app starts. The server
(tools/databricks_notebook_mcp.py) has 41;
the full profile adds catalogue and schema listings, lineage, grants, pipelines and warehouses.

Three rules the server enforces

How it works

flowchart LR
    U["You<br/>browser, Databricks SSO"] --> A["OpenGenie App<br/>FastAPI + React 19"]
    A -->|"ACP over stdio"| O["opencode agent<br/>one per conversation"]
    O -->|"MCP"| T["Tool server<br/>fenced: folder, schema, prefix"]
    T -->|"as you"| W[("Your workspace<br/>Unity Catalog, warehouses,<br/>notebooks, jobs")]
    O --> M["Your model<br/>serving endpoint / AI Gateway,<br/>OpenAI, Anthropic, Vertex"]
    A --> L[("Lakebase<br/>session journal, model catalogue,<br/>encrypted keys")]

• One process. Uvicorn serves the API and the built front end, and spawns
  opencode acp per conversation (app.yaml).
• Durable work. Turns run on the server and are journalled in Lakebase, Postgres or SQLite. Close the
  tab and come back; after a restart the app states what was interrupted rather than replaying it
  (spec 010).
• A mission view. Each request shows its objective, plan, outputs and the proof for each requirement
  (spec 011).
• A notebook, not a chat log. Cells with their own Ask and Run, Run all that stops at the first
  failure, a run map, Fix with agent that sends the engine's own error, attachments and pasted images
  (009, 010).

Genie Code or OpenGenie?

Stay on Genie Code if you want nothing to deploy or operate; one model and the metered cost are
fine; your hardest problem is finding the right table among thousands, where Genie's business semantics
help most; or you need inline completion and Python running as you on your own clusters.

Use OpenGenie if you want to choose the model per task or use one you already pay for; writes must
stay inside a named folder and schema, enforced by the server; you want "done" checked against what the
agent ran; or you want to read, change and self-host the code.

Get started

Prerequisites

• A Databricks workspace, and the Databricks CLI
  signed in with databricks auth login.
• Python 3.12 and Node.js 20 or later, to run or build the app locally.
• For a deployment: app compute, a Lakebase project, and a Databricks secret scope for the encryption key.

Deploy

The install page lists the values in app.yaml and
the Makefile that name the author's workspace; change those first. Then:

git clone https://github.com/raphaelmansuy/opengenie.git && cd opengenie
databricks bundle deploy -t dev            # the app, its user scopes and secret resources
make -C apps/coding-agent deploy           # sync -> deploy -> verify the live URL serves this checkout

Run locally

cd apps/coding-agent
make setup && make dev                     # scripted agent on http://127.0.0.1:8062
make dev-real                              # the real opencode agent and model on http://127.0.0.1:8071
make check                                 # TypeScript types, 1303 Python tests and 959 front-end tests; no network

make dev replaces the model with a deterministic agent that plays scripted scenarios, which is how the
interface is developed and tested. Both use your databricks auth login session, so the workspace browser
shows your real files.

Which version is running

The version lives in one file, apps/coding-agent/VERSION, and every surface
reads it from there: the top bar prints it beside the product name, the account menu names the commit the
platform resolved, and GET /api/version answers the same facts over HTTP. A deployed app therefore
states its own build, which is what make -C apps/coding-agent verify checks against this checkout.
Releases are tagged from that file — see Releases.

The site has its own gate. python3 site/build.py && python3 site/check.py builds and verifies every
link, claim and image; node tests/sites/e2e/site.e2e.mjs drives every published page at six viewports
in both themes and saves each screenshot to tests/sites/e2e/screenshots/. CI runs both before it
publishes.

The study behind it

The app is the second half of this repository. The first half is the measurement that justified it:
eleven graded tasks (eight to develop with, three held out), seeded in a live workspace and scored
against it. The repository keeps 174 run files; 123 are counted in the claims, and the other batches are
named as exclusions on the evidence page. Each task seeds real state
(tables with traps, a broken job, a support ticket carrying an injected instruction), hands the agent a
plain-language request, then grades the workspace afterwards. A grader is trusted only once the
reference solution scores 100% and a do-nothing agent fails.

1. The model matters more than the tooling. The same tools and rules scored 39% with one model and
   86% with another. → Results
2. Engineering rescues a weak model. Scoped tools, a supervisor loop and written rules took that
   model from 19% to 94% of the points on the development tasks (29 of 152 to 143 of 152).
3. A strong model needs far less machinery. On the three held-out tasks, the very first setup scored
   every point with Gemini (37 of 37, over three runs). Choose how much to build from the model you can afford.
   → Decision guide
4. The official Databricks skills are not neutral. One routes data questions to Genie, and the plain
   CLI path has no write fence. → Toolbox
5. Never trust a self-reported "done". The most dangerous failures were confident and wrong.
   → Exam suite

python3 exams/make_tables.py               # print every table from exams/results
python3 site/build.py && python3 site/check.py   # rebuild the site; fails if a number and its source disagree

Repository map

apps/coding-agent/   the application: FastAPI, React 19 + CodeMirror 6, opencode over ACP
  api/               domain (pure rules), ports, adapters, routers, services
  web/src/           the front end and its tests
  tools/             the MCP tool server (41 tools, 28 in the core profile) and its skills
  seed/              the model catalogue the app ships with
exams/               the exam harness: 11 graded missions and every saved run
docs/                tutorial, big picture, setup, safety, results, decision guide
specs/               one folder per feature: why, design, plan, and browser evidence
site/                the GitHub Pages site: build.py, claims.py, check.py, content/
tests/sites/         end-to-end tests and screenshots of the site
notebooks/           the tutorial as one notebook, for a workspace with no shell
templates/           the original shell-only kit

What this is not

• Not a replacement for Genie's business-term understanding. The exams named their tables; finding
  the right one in a catalogue of ten thousand is untested here.
• Not a cost comparison. We measured tokens, not money, and hold no Genie Code usage history.
• Not a hosted demo. The app needs your workspace.
• Not a guarantee. One run per setup on most configurations, small clean data, well-specified tasks.
  Only one configuration was repeated (spread about 2.6 points), so a gap of a point or two is noise.
• Not a verified deployment. This README describes the code. To check that a live app serves this
  checkout, run make -C apps/coding-agent verify.

Safety

• The identity is the boundary. Deployed, the app acts as the signed-in user; locally, as the owner
  of the token. The fence limits what the agent asks for; the identity limits what it may do.
• Agents can be prompt-injected by data. Exam M8 plants an instruction in a support ticket. Read the
  "anything odd or suspicious" section of every report.
• Never run an agent with auto-approve against production.

Read Safety and governance before running anything that can write.

Licence

Apache-2.0. Use it, fork it, run it inside a company; keep the attribution.

Author

Raphael Mansuy: data engineering and AI strategist, twenty years of global technology leadership, a
startup studio in Hong Kong, co-author of The Definitive Guide to Data Integration (Packt, 2024).

LinkedIn · GitHub ·
The book ·
About

Site: https://raphaelmansuy.github.io/opengenie/ ·
Source: https://github.com/raphaelmansuy/opengenie ·
Releases: https://github.com/raphaelmansuy/opengenie/releases ·
Issues: https://github.com/raphaelmansuy/opengenie/issues