ThinkWatch

English | 中文

The enterprise-grade secure gateway for AI. Secure, audit, and govern every AI API call and MCP tool invocation across your organization — from a single control plane.

Just as an SSH secure gateway is the single gateway through which all server access must flow, ThinkWatch is the single gateway through which all AI access must flow. Every model request. Every tool call. Every token. Authenticated, authorized, rate-limited, logged, and accounted for.

                    ┌──────────────────────────────────────┐
 Claude Code ──────>│                                      │──> OpenAI
 Cursor ───────────>│    Gateway  :3000                    │──> Anthropic
 Custom Agent ─────>│    AI API + MCP Unified Proxy        │──> Google Gemini
 CI/CD Pipeline ───>│                                      │──> Azure OpenAI / AWS Bedrock
                    └──────────────────────────────────────┘
                    ┌──────────────────────────────────────┐
 Admin Browser ────>│    Console  :3001                    │
                    │    Management UI + Admin API          │
                    └──────────────────────────────────────┘

Why ThinkWatch?

As AI agents proliferate across engineering teams, organizations face a growing governance challenge:

• API keys scattered everywhere — hardcoded in .env files, shared in Slack, rotated never
• Zero visibility — who used which model, how many tokens, at what cost?
• No access control — every developer has direct access to every model and every MCP tool
• Compliance gaps — no audit trail for AI-assisted code generation or data access
• Cost surprises — monthly AI bills that nobody can explain or attribute

ThinkWatch solves all of this with a single deployment.

Key Features

AI API Gateway

• Multi-format API proxy — natively serves OpenAI Chat Completions (/v1/chat/completions), Anthropic Messages (/v1/messages), and OpenAI Responses (/v1/responses) APIs on a single port; works as a drop-in replacement for Cursor, Continue, Cline, Claude Code, and the OpenAI/Anthropic SDKs
• Multi-provider routing — OpenAI, Anthropic, Google Gemini, Azure OpenAI, AWS Bedrock, or any OpenAI-compatible endpoint
• Automatic format conversion — Anthropic Messages API, Google Gemini, Azure OpenAI, AWS Bedrock Converse API, and more, all behind a unified interface
• Provider auto-loading — active providers are loaded from the database at startup and registered in the model router; default model prefixes (gpt-/o1-/o3-/o4- for OpenAI, claude- for Anthropic, gemini- for Google) route automatically; Azure and Bedrock require explicit model registration
• Streaming SSE pass-through — zero-overhead forwarding with real-time token counting
• Virtual API keys — issue scoped tw- keys per team, per project, per developer; revoke in one click
• API key lifecycle management — automatic rotation with grace periods, per-key inactivity timeout, expiry warnings, and background policy enforcement
• Sliding-window rate limiting — RPM and TPM limits via Redis, per key or per user
• Circuit breaker — three-state (Closed/Open/HalfOpen) circuit breaker with configurable failure threshold and recovery period
• Retry with exponential backoff — configurable retries with jitter for network errors and upstream rate limits
• Real-time cost tracking — per-model pricing with budget alerts and team attribution

MCP Gateway

• Centralized tool proxy — one MCP endpoint that aggregates tools from all upstream servers
• Namespace isolation — github__create_issue, postgres__query — no tool name collisions
• Tool-level RBAC — control exactly which users or roles can invoke which tools
• Connection pooling & health monitoring — automatic reconnection, background health checks
• Full audit trail — every tool invocation logged with user, parameters, and response

Security & Compliance

• Dual-port architecture — gateway (public-facing) and console (internal-only) on separate ports
• Role-based access control — 5-tier RBAC: Super Admin, Admin, Team Manager, Developer, Viewer
• SSO/OIDC — plug into Zitadel, Okta, Azure AD, or any OIDC-compliant provider
• AES-256-GCM encryption — provider API keys and secrets encrypted at rest
• SHA-256 key hashing — virtual API keys stored as hashes; plaintext shown exactly once
• Content Security Policy — CSP headers on the console port to prevent XSS and injection attacks
• JWT entropy enforcement — minimum 32-character secret with entropy validation at startup
• Startup dependency validation — verifies PostgreSQL, Redis, and encryption key availability with clear error messages before accepting traffic
• Security headers — X-Content-Type-Options, X-Frame-Options, CORS whitelisting, request timeouts
• Soft-delete — users, providers, and API keys use soft-delete (deleted_at column) with automatic purge after 30 days
• Password complexity — minimum 8 characters with required uppercase, lowercase, and digit
• Session IP binding — admin sessions bound to client IP; stolen tokens cannot be replayed from a different network
• Distroless containers — minimal attack surface in production (2MB runtime image, no shell)

Operations & Configuration

• Dynamic configuration — most settings stored in database (system_settings table), configurable via Web UI (Admin > Settings with 7 category tabs)
• First-run setup wizard — guided /setup wizard creates the super_admin account, configures the site, and optionally adds the first provider and API key
• Configuration Guide — built-in /gateway/guide page in the web console with copy-paste setup instructions for Claude Code, Cursor, Continue, Cline, OpenAI SDK, Anthropic SDK, and cURL; auto-detects the gateway URL
• Multi-instance sync — configuration changes propagated across instances via Redis Pub/Sub
• Data retention policies — configurable retention periods for usage records and audit logs with automatic daily purge

Observability

• Prometheus metrics — GET /metrics endpoint on the gateway port (3000) exposing gateway_requests_total, gateway_request_duration_seconds, gateway_tokens_total, gateway_rate_limited_total, circuit_breaker_state, and more
• Enhanced health checks — /health/live (liveness probe), /health/ready (readiness probe with PostgreSQL and Redis checks), /api/health (detailed latency and pool statistics)
• ClickHouse-powered audit logs — SQL-queryable audit logs across all API calls and tool invocations, stored in ClickHouse for high-performance columnar analytics
• Audit log forwarding — multi-channel delivery: UDP/TCP Syslog (RFC 5424), Kafka, and HTTP webhooks — route audit events to any SIEM, data lake, or alerting pipeline
• Usage analytics — token consumption by user, team, model, and time period
• Cost analytics — MTD spend, budget utilization, per-model cost breakdown
• Health dashboard — real-time status of PostgreSQL, Redis, ClickHouse, and all MCP servers
• Unified log explorer — search across audit, gateway, MCP, access, and platform logs from a single page with structured query syntax

Tech Stack

Quick Start

# 1. Start infrastructure
make infra

# 2. Start backend (gateway :3000 + console :3001)
cp .env.example .env
make dev-backend

# 3. Start frontend dev server
cd web && pnpm install && pnpm dev

# 4. Complete the setup wizard at http://localhost:5173/setup

See the Deployment Guide for production setup with Docker Compose or Kubernetes.

Documentation

Full documentation: thinkwat.ch/docs

Port Architecture

In production, only port 3000 should be reachable from the internet. Port 3001 should be restricted to your admin network.

Project Structure

ThinkWatch/
├── crates/
│   ├── server/          # Dual-port Axum server (gateway + console)
│   ├── gateway/         # AI API proxy: routing, streaming, rate limiting, cost tracking
│   ├── mcp-gateway/     # MCP proxy: JSON-RPC, tool aggregation, access control
│   ├── auth/            # JWT, OIDC, API key, password hashing, RBAC
│   └── common/          # Config, DB, models, crypto, validation, audit logger
├── migrations/          # 12 PostgreSQL migration files
├── web/                 # React frontend — ~20 page components
├── deploy/
│   ├── docker/          # Dockerfile.server (distroless), Dockerfile.web (nginx)
│   ├── docker-compose.yml       # Production deployment
│   ├── docker-compose.dev.yml   # Development (PG + Redis + ClickHouse + Zitadel)
│   └── helm/think-watch/      # Kubernetes Helm chart
└── ...

Documentation: thinkwat.ch/docs

Contributing

Contributions are welcome. Please open an issue to discuss before submitting a PR for major changes.

License

ThinkWatch is source-available under the Business Source License 1.1.
Non-production use is free. Production use is free up to both 10,000,000
Billable Tokens and 10,000 MCP Tool Calls per UTC calendar month; above
either threshold, a commercial license is required and priced by usage tiers.

See LICENSING.md for the production-use thresholds, the
Billable Token and MCP Tool Call definitions, the tiering model, and the
changeover to GPL-2.0-or-later.

Star History