watchtower

Self-hosted error tracking that speaks Sentry and AppSignal.
Keep the SDK you already use and point it at your own server.

Quick start · Connect your apps · Features · Configuration · Contributing

---

Watchtower implements the ingestion protocols of hosted error trackers, so
their official SDKs report to it unchanged. Moving off a hosted tracker means
changing one setting, the DSN or the push endpoint, rather than
re-instrumenting every service. Errors are grouped into issues you can triage
with your team, alerted to Slack, Linear and email, and handed to coding
agents through MCP.

It runs as a single Go binary with Postgres as its only dependency.

Contents

• Features
• Quick start
• Connect your apps
• Triage
• Alerts and integrations
• Coding agents (MCP)
• JavaScript source maps
• Configuration
• Running in production
• How it works
• Data handling
• CLI
• Contributing
• License

Features

• Drop-in ingestion. Sentry SDKs in any language, the AppSignal agents for
  Elixir, Ruby, Node.js and Python, and the AppSignal browser SDK. Each vendor
  protocol is an adapter; everything behind the adapters is shared.
• Grouping you can trust. Errors group by their cause, with versioned
  rules that stay stable across builds, minified code and recursion.
  Resolved issues that happen again reopen as regressions.
• Triage together. Owners, comments, an activity timeline, bulk actions,
  search and filters, and releases linked to their commits.
• Alerts where you work. Slack (incoming webhook or bot token), Linear
  issues that resolve here when they're completed in Linear, and personal email
  notifications and digests through any SMTP provider.
• Built for coding agents. An MCP server gives Claude Code, Codex, Cursor
  and other agents a complete brief of an issue, with source context, and lets
  them record their work.
• JavaScript source maps uploaded with sentry-cli or Sentry's bundler
  plugins.
• Private by design. An allowlisted event model and redaction of card
  numbers, tokens and secrets before anything is stored.
• Simple to run. One binary, Postgres, health and readiness endpoints,
  Prometheus metrics, retention, and horizontal scaling.

Quick start

You need Docker with Compose. Run:

curl -fsSL https://raw.githubusercontent.com/olucurious/watchtower/main/install.sh | sh

It sets Watchtower up in ./watchtower with generated secrets, starts it with
Postgres, and prints the address and a one-time setup code. To use another
folder, run … | WATCHTOWER_DIR=my-folder sh; the installer never takes over a
folder that belongs to another project. Open the address,
enter the code, and create your administrator account. Then create a project;
its page walks you through creating a key, adding the SDK and sending a first
error, and links to it as soon as it arrives.

Images are published for amd64 and arm64. To run a specific version, set
WATCHTOWER_VERSION (for example WATCHTOWER_VERSION=0.1.0) when you run the
installer, or in watchtower/.env later.

Installing by hand, or building from source

mkdir watchtower && cd watchtower
curl -fsSLO https://raw.githubusercontent.com/olucurious/watchtower/main/compose.yaml
cat > .env <<EOF
WATCHTOWER_DB_PASSWORD=$(openssl rand -hex 16)
WATCHTOWER_SECRET_KEY=$(openssl rand -hex 32)
EOF
docker compose up -d
docker compose exec watchtower watchtower setup-code   # the code for the first account

To build the image from a checkout instead of pulling it:

git clone https://github.com/olucurious/watchtower.git && cd watchtower
# create .env as above, then:
docker compose -f compose.yaml -f compose.build.yaml up -d --build

WATCHTOWER_SECRET_KEY encrypts the credentials of Slack, Linear and other
destinations. Keep it safe and keep it stable: changing it makes those stored
credentials unreadable. To use another host port, set WATCHTOWER_PORT; for
anything beyond a local trial, see Running in production.

Connect your apps

Create a key on the project's page for the SDK the service already uses. The
key's settings are shown once; Watchtower stores only a hash.

Sentry SDKs

Replace the DSN; nothing else changes.

SENTRY_DSN=https://<key>@watchtower.example.com/<project-id>

Setting up a Sentry SDK from scratch

Elixir ({:sentry, "~> 13.4"})

# config/runtime.exs
config :sentry, dsn: System.get_env("SENTRY_DSN"), environment_name: config_env()

# lib/my_app/application.ex, in start/2: report crashed processes, and say
# where log-reported errors (such as failed database connections) came from
:logger.add_handler(:sentry_handler, Sentry.LoggerHandler, %{config: %{metadata: [:mfa, :application]}})

Node.js (npm install @sentry/node)

const Sentry = require("@sentry/node")
Sentry.init({ dsn: process.env.SENTRY_DSN, environment: process.env.NODE_ENV })

Python (pip install sentry-sdk)

import os, sentry_sdk
sentry_sdk.init(dsn=os.environ["SENTRY_DSN"], environment="production")

Browser (npm install @sentry/browser)

import * as Sentry from "@sentry/browser"
Sentry.init({ dsn: "https://<key>@watchtower.example.com/<project-id>" })

AppSignal (Elixir, Ruby, Node.js, Python)

Set two variables. Packages and instrumentation stay as they are.

APPSIGNAL_PUSH_API_ENDPOINT=https://watchtower.example.com
APPSIGNAL_PUSH_API_KEY=<key>

For the AppSignal browser SDK, create an AppSignal (browser) key and pass
its settings to new Appsignal({ key, uri }).

Tested SDKs

Adapter	Tested with
sentry	sentry-python 2.71.0, @sentry/node 11.2.0, sentry-elixir 11.0.4; source map uploads from sentry-cli 3.8.0
appsignal	Elixir 2.9.2–2.18.0 (17 releases), Ruby 5.0.1, Node.js 3.9.1, Python 1.9.0
appsignal-frontend	@appsignal/javascript 1.6.1

Every tested version has a captured payload in the test suite. The AppSignal
protocol is reverse-engineered, so that adapter is marked experimental; see
docs/adapters/appsignal.md. A weekly workflow
checks for new AppSignal releases.

Triage

• Issues filter by project, environment, owner and period, with search
  (/), status tabs, sparklines and bulk resolve, mute or reopen.
• An issue shows the exception chain with your own frames first and source
  context around each line, highlights, how events vary by release,
  environment and server, and charts for the last 24 hours and 30 days.
  Errors reported from a log line show where they were logged.
• Ownership and discussion: assign an issue, comment on it, and follow its
  timeline of status changes, regressions, alerts and links.
• Releases that are commit SHAs link to the commit once the project names
  its repository.
• On phones, the summary comes first and everything stays usable.

Alerts and integrations

Alerts fire when an issue is new, when a resolved issue comes back, or when an
issue reaches a number of events in an hour, optionally limited to a minimum
level and one environment. They're written to an outbox in the same
transaction as the event that triggered them, then delivered with retries, so
an event is never stored without its alert. Each destination shows its last
delivery or error and has a Test button.

Slack

Use an incoming webhook, or an existing Slack app's bot token (xoxb-…) with
a channel ID and the bot invited to that channel. Messages link straight to
the event that triggered them.

Linear

Add a Linear destination with an API key (Linear › Settings › Security &
access › Personal API keys) and pick a team.

• Create Linear issue on an issue's page files it with the error, its
  location, the top stack frames, counts, release and a link back. The issue
  shows its Linear identifier and state from then on.
• Alert rules can also file issues automatically. An issue already in Linear
  gets a comment instead of a duplicate. With no rules, the destination is
  used only by hand.
• Completing the Linear issue resolves it here. Watchtower polls Linear
  every five minutes instead of receiving webhooks, so it works on a private
  network.

Each Linear issue is created with an ID derived from the Watchtower issue and
your secret key, so a retried or concurrent request can never file it twice.

Email

Members get email when an issue is assigned to them, when an issue they own
regresses, and when someone comments on one, and can opt in to a daily or
weekly digest of new issues, regressions and the busiest issues. Each member
chooses on their Account page, which also has a Send test email button
that shows the mail server's answer.

Set WATCHTOWER_EMAIL_PROVIDER to turn email on:

• smtp works with any provider: Postmark, Amazon SES, Mailgun, Resend,
  Google Workspace, Cloudflare Email Service, or a relay of your own.
• cloudflare uses Cloudflare Email Service's REST API, which also accepts
  user-owned tokens (its SMTP endpoint accepts only account-owned ones).

Provider	Host	Port and TLS	Username
Postmark	smtp.postmarkapp.com	587, starttls	the server API token (also the password)
Amazon SES	email-smtp.<region>.amazonaws.com	587, starttls	SMTP credentials from the SES console
Cloudflare Email Service	smtp.mx.cloudflare.net	465, tls	api_token; the password is an account-owned API token with Email Sending: Edit
Local relay (Postfix, …)	its address	25, none	usually none

The sender's domain must be verified with your provider.

Coding agents (MCP)

Watchtower runs a Model Context Protocol
server at /mcp, so an agent can pick up an error, understand it and fix it.
Create a token under Account › Agent access; the dialog shows the setup for
Claude Code, Codex, Cursor and other clients. For Claude Code:

claude mcp add --transport http watchtower https://watchtower.example.com/mcp \
  --header "Authorization: Bearer wtp_…"

Then ask, for example, "Find the most frequent unresolved error in the
reader project in Watchtower and fix it."

Tool	What it does
list_projects	Projects with unresolved counts and recent volume
list_issues	Find issues by project, status, text, environment, owner or period
get_issue	The brief above: stack trace with source context, cause chain, frequency, releases, how events vary, and the team's comments
list_events, get_event	Individual occurrences in full
assign_issue, add_comment, update_issue_status, create_linear_issue	Record work; these need a write token

Tokens act as the member who created them, so an agent's changes appear under
that member's name. They are read-only or read-write, can expire after 30 or
90 days, are stored only as SHA-256 digests, and stop working when revoked or
when their owner is disabled. Error text reported by applications is labelled
as untrusted data in every brief, and the endpoint rejects cross-origin
browser requests.

JavaScript source maps

Watchtower accepts uploads from sentry-cli and Sentry's Vite, webpack and
esbuild plugins. Create an upload token on the project's page, then in CI:

export SENTRY_URL=https://watchtower.example.com SENTRY_AUTH_TOKEN=wtk_… SENTRY_ORG=watchtower SENTRY_PROJECT=web
npx @sentry/cli sourcemaps inject ./dist
npx @sentry/cli sourcemaps upload ./dist

Before grouping, minified frames are mapped back to the original file,
function and line, with surrounding source. Bundles match by debug ID, or by
release and file URL for uploads without one, so issues group by your
original code and stay stable across builds.

Configuration

Watchtower reads WATCHTOWER_* environment variables.

Variable	Default	Purpose
WATCHTOWER_DATABASE_URL	required	Postgres connection URL. Each process opens up to 25 connections; set pool_max_conns in the URL to change that
WATCHTOWER_PUBLIC_URL	http://localhost:8080	Address SDKs and browsers use; printed in DSNs and links. With https://, session cookies are marked Secure
WATCHTOWER_SECRET_KEY	none	32 bytes, hex or base64. Encrypts destination credentials at rest; required for Slack and Linear
WATCHTOWER_LISTEN	:8080	HTTP listen address
WATCHTOWER_ADAPTERS	sentry,appsignal,appsignal-frontend	Enabled adapters
WATCHTOWER_ROLES	ingest,worker	Serve ingestion, run the background workers, or both
WATCHTOWER_RETENTION_DAYS	90	Older events, issues with no recent occurrences, source maps and alert and email history are deleted hourly
WATCHTOWER_MAX_BODY_BYTES	20 MiB	Request size on the wire
WATCHTOWER_MAX_DECOMPRESSED_BYTES	50 MiB	Request size after decompression
WATCHTOWER_MAX_EVENT_BYTES	1 MiB	Size of one event
WATCHTOWER_MAX_QUEUE_DEPTH	100000	Queued events before SDKs are told to back off. Approximate: each process recounts the queue every second
WATCHTOWER_WORKER_BATCH_SIZE	100	Events per worker transaction
WATCHTOWER_SLACK_WEBHOOK_HOSTS	hooks.slack.com	Hosts that Slack webhooks may point at
WATCHTOWER_EMAIL_PROVIDER	smtp when WATCHTOWER_SMTP_HOST is set, otherwise off	smtp or cloudflare
WATCHTOWER_EMAIL_FROM	required with email	Sender, such as Watchtower <[email protected]>
WATCHTOWER_SMTP_HOST	none	Mail server for the smtp provider
WATCHTOWER_SMTP_PORT	587, or 465 with tls	Mail server port
WATCHTOWER_SMTP_TLS	starttls, or tls on port 465	tls (implicit), starttls (required, never downgraded) or none (a trusted relay; credentials are only ever sent unencrypted to localhost)
WATCHTOWER_SMTP_USERNAME, WATCHTOWER_SMTP_PASSWORD	none	SMTP credentials
WATCHTOWER_CLOUDFLARE_ACCOUNT_ID, WATCHTOWER_CLOUDFLARE_API_TOKEN	none	For the cloudflare email provider
WATCHTOWER_LOG_LEVEL	info	debug, info, warn or error

Invalid settings stop Watchtower at startup with a message naming the
variable, rather than failing later.

Running in production

• TLS and a stable address. Put Watchtower behind a reverse proxy that
  terminates TLS, and set WATCHTOWER_PUBLIC_URL to the address SDKs and
  people use. SDKs need to reach the ingestion endpoints; the web UI and
  /mcp can stay on a private network.
• Upgrades. Run the installer again, or docker compose pull && docker compose up -d in the install directory. Database migrations run
  automatically at startup.
• Backups. Everything lives in Postgres; back it up with pg_dump like
  any other database, and keep WATCHTOWER_SECRET_KEY with it.
• Health and metrics. GET /healthz checks the process; GET /readyz
  also checks the database. GET /metrics serves Prometheus counters for
  accepted, rejected and shed events per adapter, worker outcomes, alerts,
  emails and retention.
• Scaling. Run more replicas, or split ingestion and background work with
  WATCHTOWER_ROLES. Workers coordinate through Postgres SKIP LOCKED and
  lock issues in a fixed order, so any number can run at once. Each process
  uses up to 25 database connections (pool_max_conns in the URL); keep
  replicas × pool size below Postgres's max_connections.
• Throughput. On one 6-core machine shared with Postgres, a single
  process accepted about 2,000 events a second, and one worker stored about
  2,000 a second; an error storm on one issue is the cheapest case, because
  a batch updates each issue once.
• Back-pressure. When the queue is full, SDKs get the vendor's own
  back-off response (for example 429 with X-Sentry-Rate-Limits) instead of
  events being dropped silently. The limit is approximate: replicas together
  can pass it by about a second of traffic. Events that fail processing five
  times move to ingest_dead_letter.

How it works

Sentry SDKs      ─▶ sentry adapter              ─┐
AppSignal agents ─▶ appsignal adapter           ─┼─▶ scrub ─▶ queue ─▶ worker ─▶ issues & events
Browser SDK      ─▶ appsignal-frontend adapter  ─┘                  (symbolicate,     │
                                                                     group, alert)    ├─▶ web UI and API
                                                                                      ├─▶ Slack · Linear · email
                                                                                      └─▶ MCP for coding agents

1. An adapter authenticates the request, decodes the vendor's format and
   converts it to Watchtower's event model. Adapters only translate.
2. Events are scrubbed and written to a durable queue in Postgres. The SDK
   gets its success response only after that write.
3. A worker maps minified frames through source maps, groups the event
   into an issue, detects regressions and writes any alerts and emails, all in
   one transaction.
4. Separate loops deliver alerts and emails with retries, sync Linear, and
   apply retention.

Data handling

• Allowlist, not blocklist. Request bodies, headers, cookies, breadcrumbs
  and process state sent by SDKs are dropped at the adapter. Of the free-form
  "extra" data, only the logger metadata an app explicitly sends and a failed
  job's worker, queue and attempt are kept. User data is reduced to an opaque
  ID.
• Redaction. Before storage, card numbers (Luhn-valid), bearer tokens and
  secret-looking key=value pairs are redacted, as are tags, context values
  and URL parameters with sensitive names.
• No sensitive logging. Request bodies and query strings are never logged.
• Credentials at rest. Keys and tokens are stored as SHA-256 digests;
  Slack and Linear credentials are encrypted with AES-256-GCM.
• Errors only. Transactions, sessions, profiles, replays, logs and metrics
  are acknowledged and counted but not stored. Watchtower is an error tracker,
  not an APM.

CLI

watchtower setup-code                           # a new code for creating the first account in the browser
watchtower user create [email protected] -admin   # the password is prompted, never in argv
watchtower project create reader -name "Reader Web"
watchtower key create reader sentry
watchtower issues reader -status unresolved
watchtower issue resolve 12 -actor ada
watchtower adapters                             # adapters and tested SDK versions
watchtower healthcheck                          # exits 0 when the local server is healthy

Contributing

Contributions are welcome. CONTRIBUTING.md covers setting
up a development environment, running the tests and preparing a pull request,
and AGENTS.md describes the architecture and the rules every
change must keep, for people and coding agents alike.

License

Watchtower is licensed under the Apache License, Version 2.0.
Contributions are accepted under the same licence.