Airlock

AI release engineering.

GitHub Actions for safely shipping AI agents - CI release gate for prompts, skills, MCP, and models. Not another eval platform.

Airlock is the release-control layer for AI systems: it treats models, prompts, tools, skills, MCP servers, judges, and eval sets as a releasable unit, detects what changed, evaluates behavior against policy with statistical confidence, and blocks or approves the ship - including when the change came from upstream, not from your commit.

The open-source distribution is a local-first Go toolchain (binary + CI Action + .airlock/ store). Prove that gate in CI first; the hosted control plane comes later (open-core).

First public beta · no telemetry · Apache-2.0 · state under .airlock/ · versions in CHANGELOG
APM tells you what the agent depends on · eval platforms tell you how it scored · Airlock tells you whether that change is safe to ship.
Not a Promptfoo/Braintrust replacement - the release gate beside them.

The problem

Traditional software has a release pipeline. AI systems change behavior without a conventional code change: a provider updates a model behind a stable string, a prompt edits, a skill lands, a tool schema widens, an MCP server gains permissions, a retrieval index drifts, a judge shifts.

The production question is: can we safely release this AI change?
And its mirror: did a change we never made just get released to us?

How it works

flowchart LR
  edit["Edit prompt / skill / MCP"]
  snap["Snapshot"]
  diffNode["Diff + blast radius"]
  policy["Policy: PASS / FAIL / NEEDS_APPROVAL"]
  ship["Ship or block in CI"]
  edit --> snap --> diffNode --> policy --> ship

State lives under .airlock/ in your application repo. Nothing uploads by default.

Who it’s for

Not yet: hosted control plane (Phase 5), enterprise SSO / EU / K8s (Phase 6), release agent (Phase 7). See docs/ROADMAP.md.

Install

Platforms: Linux / macOS · amd64 / arm64 (Windows not yet).
Pin a pre-release tag from Releases (GitHub “latest” skips them). Current tag: see CHANGELOG.

curl -sSL https://raw.githubusercontent.com/ankittk/airlock/main/install.sh | AIRLOCK_VERSION=v0.1.0-beta.6 bash
# or: go install github.com/ankittk/airlock/cmd/[email protected]

Quick start - break a prompt, watch Airlock catch it

Clone this repo. Toy agent at testdata/toy-agent ships prompts, a skill, MCP, and replay cassettes - no API keys.

cd testdata/toy-agent
airlock init && airlock snapshot

$ airlock init && airlock snapshot
Wrote .airlock/manifest.json
  agents=1 models=1 prompts=2 tools=0 skills=1 mcp=2 evals=5
snapshot abc123…  artifacts=12  manifest=def456…

Nudge the system prompt (or edit a skill / widen MCP permissions):

echo "You are a DIFFERENT support agent." >> prompts/system.md
airlock snapshot && airlock diff

$ airlock snapshot && airlock diff
snapshot ghi789…  artifacts=12  manifest=jkl012…

Changed AI artifacts:
  ~ prompt:system-prompt (a1b2c3d4 → e5f6a7b8)

Blast radius - agents: support-bot
Blast radius - evals:  default

Run cheap replay evals and emit the PR body:

airlock test --mode replay
airlock ci --comment

$ airlock test --mode replay
Verdict: PASS
metric             rate             95% CI  gate
task_success      100.0%  [ 47.8%, 100.0%]  PASS
samples=9 cost=$0.0000  (cassette replay)

$ airlock ci --comment

### Airlock
This PR changes AI artifacts:
- `changed` **prompt:system-prompt**

Blast radius: agents **support-bot**

### Airlock eval

**Verdict: PASS**

| metric | rate | 95% CI | gate | reason |
|---|---:|---|---|---|
| `task_success` | 100.0% | [47.8%, 100.0%] | **PASS** | no significant regression (delta CI [-0.02, 0.02]) |

Flip the story - expand MCP power instead of a prompt:

# e.g. add "write" under local-fs permissions in apm.lock.yaml
airlock snapshot && airlock diff
airlock ci --comment --fail-on-approval

$ airlock diff
Changed AI artifacts:
  ~ mcp:local-fs (… → …)
NEEDS_APPROVAL: MCP new permission write on local-fs

$ airlock ci --comment --fail-on-approval
### Airlock
…
**NEEDS_APPROVAL:** MCP new permission write on local-fs

Run `airlock approve --base … --head …` to unblock.
exit 1   # merge blocked until approved

Flip it again - a prompt edit that quietly rides in with a new dependency (agent-driven supply chain):

# edit prompts/system.md AND add a package under `packages:` in apm.lock.yaml
airlock diff --base <baseline-snapshot-id>
airlock ci --base <baseline-snapshot-id> --fail-on-approval

$ airlock diff --base aaee8c11172c86b1
Changed AI artifacts:
  + dependency:left-pad
  ~ prompt:system-prompt (a68e98e0 → 31125b19)
Blast radius - agents: support-bot
NEEDS_APPROVAL: new dependency: left-pad

$ airlock ci --base aaee8c11172c86b1 --fail-on-approval
…
airlock ci: NEEDS_APPROVAL without ledger entry (run: airlock approve --base … --head …)
exit 1   # merge blocked until: airlock approve --base … --head …

A dependency added on its own (no prompt/skill/MCP/agent change alongside it) does not trigger this - that PR is Dependabot / SCA's job, not Airlock's. Details: docs/ROADMAP.md.

That is the company wedge: AI change control on the PR, not “hope the prompt looks fine.”

--mode live hits real providers (API keys + budgets.max_cost_per_pr). Full walkthrough: docs/GUIDE.md.

Use it on your repo

1. In your application repo: airlock init → commit .airlock/policy.yml (and keep snapshots as you prefer).
2. Copy .github/workflows/airlock.yml into that repo (not into this one).
3. Company default: fail closed with --fail-on-approval / --fail-on-eval (sample workflow defaults AIRLOCK_FAIL_ON_APPROVAL=true).

Security in CI

Airlock gates AI release risk on the PR - not general AppSec:

Skill / MCP power expansion → NEEDS_APPROVAL. Approvals are advisory until CI uses --fail-on-approval.

Supply chain (npm, crates.io, PyPI, …): classic malware-in-the-lockfile is still Dependabot / SCA / provenance. Agents make it worse by proposing or merging deps at machine speed. Airlock’s angle is the AI release surface: when a prompt/skill/MCP/agent change also expands an APM-tracked package dependency, that lands in blast radius as NEEDS_APPROVAL (--fail-on-approval blocks merge) - a dep-only PR with no AI-artifact change is left to SCA. Not replacing package-manager security scanners. Details: docs/ROADMAP.md.

If you use LangSmith (or Braintrust / Langfuse / Phoenix)

Keep the observability + eval platform. Airlock is the release gate beside it, not a replacement.

Today (beta):

1. Keep tracing and datasets in LangSmith (or similar).
2. In the application repo: airlock init, point evals at cases you already trust (Promptfoo YAML, or export dataset → Airlock eval JSONL / airlock import promptfoo).
3. Tune .airlock/policy.yml thresholds; add the sample workflow with --fail-on-eval / --fail-on-approval.
4. Optional: feed production signal via airlock ingest otel → baseline / drift (OTel JSONL; not a live LangSmith API sync yet).

Not yet: native LangSmith connector, prompt playground, hosted annotation queues, managed agent deploy. Those stay on their platform; Airlock borrows the flexibility into later phases without becoming the trace UI. Details: docs/ROADMAP.md.

What ships in the stack

A full AI release stack, not a single command:

flowchart TB
  inputs["git + APM + prompts + skills + MCP + OTel"]
  engine["Airlock release engine"]
  out["CI decision + gateway routing hints"]
  inputs --> engine --> out
  subgraph parts [Inside the engine]
    discovery["discovery / manifest / snapshot / diff"]
    evals["evals + stats + cassettes + judges"]
    pol["policy → ship / block / approve"]
    store["local .airlock store"]
  end
  engine --- discovery
  engine --- evals
  engine --- pol
  engine --- store

What init can see today

airlock init is not every industry SDK:

Agent dependency locking is APM’s job; Airlock imports it. Details: GUIDE - discovery.

Commands

Gates fire only when a confidence interval excludes the threshold. Cassettes replay identical provider calls by request hash (not Docker layers).

Status & roadmap

OSS release gate (now)  →  prove in CI  →  Phase 5 team plane  →  Phase 6 enterprise  →  Phase 7 release agent

Details: docs/ROADMAP.md - thesis, open-core, integration maps, explicit non-goals (incl. “not code test selection”).

Design-partner outreach continues (process, not a phase). Release notes: CHANGELOG.md.

Development

go test ./... -count=1 -race
golangci-lint run ./...

This repository’s CI is .github/workflows/ci.yml (Go test/lint). Releases: docs/RELEASING.md.

Contributing · Security · Support · Changelog · Guide · Roadmap

License

Apache License 2.0