Inja LLM Gateway

Small, dependency-free LLM API gateway
OpenAI · Anthropic · Gemini dialects · multi-provider routing · usage hooks
One static binary — laptop, Docker, or Kubernetes

Documentation · Quickstart · HTTP API · Config · Deploy · Contributing · Security · License

---

Clients speak OpenAI, Anthropic, or native Gemini. The gateway routes to any configured upstream (OpenAI, Anthropic, Google, DeepSeek, xAI, Moonshot, OpenRouter, vLLM, …), passthroughs same-family traffic, translates cross-dialect chat when needed, and emits one usage event per request (JSONL / webhook / Go hook). Stateless — no database.

  OpenAI SDK / Anthropic SDK / Gemini client / Claude Code / curl
                            │
                            ▼
                    ┌───────────────┐
                    │  llm-gateway  │──► usage hooks (JSONL / webhook)
                    └───────┬───────┘
                            │
         ┌──────────┬───────┼────────┬────────────┐
         ▼          ▼       ▼        ▼            ▼
      OpenAI   Anthropic  Google  OpenAI-compat  …
                              native   (xAI, DeepSeek, …)

Stateless	No DB, sessions, or sticky routing — scale identical replicas
Cloud-native	Distroless Docker, K8s sample, SIGTERM drain, env overrides
Local-first	Single binary on macOS, Linux, Windows; docker compose up
Deps	Runtime: gopkg.in/yaml.v3 only
Module	github.com/inja-online/llm-gateway
License	AGPL-3.0

Docs site: inja-online.github.io/llm-gateway ·
Also in-repo: compatibility matrix · SDK hermetic matrix · deprecation policy · Claude Code checklist · multipart security · CHANGELOG

---

Table of contents

• Quickstart
• Features
• HTTP API
• Model routing
• Configuration
• Auth & keys
• Provider notes
• Passthrough vs translation
• Hooks & usage events
• Claude Code
• Library use
• Architecture
• Deploy
• Development & CI
• Roadmap
• Contributing
• License

---

Quickstart

1. Binary

git clone https://github.com/inja-online/llm-gateway.git
cd llm-gateway
go build -o llm-gateway ./cmd/gateway

cp gateway.example.yaml gateway.yaml
# edit providers / keys / hooks
./llm-gateway -config gateway.yaml
# override: GATEWAY_CONFIG=…  GATEWAY_LISTEN=0.0.0.0:8787

curl -s http://localhost:8787/healthz
# {"status":"ok"}

2. Docker

docker compose up --build
# or
docker build -t llm-gateway:local .
docker run --rm -p 8787:8787 \
  -e OPENAI_API_KEY -e ANTHROPIC_API_KEY -e GEMINI_API_KEY \
  -v "$PWD/gateway.yaml:/config/gateway.yaml:ro" \
  llm-gateway:local

3. Kubernetes

kubectl apply -f deploy/k8s/gateway.yaml
# point the Deployment image at your registry build

Minimal config

listen: ":8787"

providers:
  openai:
    kind: openai
    base_url: "https://api.openai.com/v1"
    api_key_env: OPENAI_API_KEY
  anthropic:
    kind: anthropic
    base_url: "https://api.anthropic.com/v1"
    api_key_env: ANTHROPIC_API_KEY
  google:
    kind: google
    base_url: "https://generativelanguage.googleapis.com/v1beta"
    api_key_env: GEMINI_API_KEY
  deepseek:
    kind: openai_compat
    base_url: "https://api.deepseek.com"

defaults:
  openai_dialect: openai
  anthropic_dialect: anthropic
  google_dialect: google

aliases:
  fast: deepseek/deepseek-chat

hooks:
  jsonl:
    output: stdout

Full commented sample: gateway.example.yaml.

---

Features

Area	What you get
Triple ingress	OpenAI Chat Completions, Anthropic Messages (Claude Code), native Gemini generateContent
Multi-provider egress	openai, openai_compat, anthropic, google
Passthrough-first	Same dialect → near-verbatim bytes
Cross-dialect chat	OpenAI ↔ Anthropic ↔ Google translation (structured outputs, thinking, tools, …)
Media	Images, video jobs, TTS/STT (OpenAI paths + Anthropic-version gate + Google speech)
Agent surface	Responses, Files (OpenAI + Anthropic), Message Batches, Moderations
Realtime	OpenAI Realtime WS + Google Live passthrough; cross-protocol bridge not implemented (fail-closed)
Usage metering	JSONL, async webhook, or in-process Go hook — one event per proxied request
Ops	One YAML file, /healthz, body limits, optional edge auth, multi-arch releases

---

HTTP API

Chat

Method	Path	Notes
POST	/v1/chat/completions	OpenAI dialect (also Gemini OpenAI-compat clients)
GET / POST / DELETE	/v1/chat/completions · /{id}	Stored completions (store=true) openai-family proxy
POST	/v1/messages	Anthropic dialect (anthropic-version required by clients)
POST	/v1/messages/count_tokens	Anthropic proxy, Google :countTokens map, or local estimate
POST	/v1beta/models/{model}:generateContent	Native Gemini
POST	/v1beta/models/{model}:streamGenerateContent	Native Gemini SSE (?alt=sse upstream)
POST	/v1beta/models/{model}:countTokens	Native Gemini; no usage event

Discovery & health

Method	Path	Notes
GET	/v1/models	Config-derived catalog + capabilities; live Anthropic when anthropic-version or ?live=1
GET	/v1/models/{id}	Single entry (provider/model ids); live Anthropic with same triggers
GET	/v1beta/models	Gemini models list passthrough (?provider= or defaults.google_dialect)
POST / GET / DELETE	/v1beta/fileSearchStores…	Gemini File Search stores (kind:google)
POST / GET / PATCH / DELETE	/v1beta/cachedContents…	Gemini context cache CRUD (kind:google); reference via cachedContent on generateContent
POST / GET / DELETE	/v1beta/tunedModels…	Gemini tuned model lifecycle (kind:google)
GET	/v1beta/models/{model}	Gemini model get / Live upgrade when :bidiGenerateContent
GET	/healthz	{"status":"ok"} — process liveness only
GET	/v1/health/providers	Optional upstream probes when health_checks.enabled (default off)
GET	/metrics	Prometheus via client_golang/promhttp (counters + histogram + Go/process; open with edge auth)

Embeddings, Responses, Files, Batches

Method	Path	Notes
POST	/v1/embeddings	OpenAI-family passthrough; OpenAI→Google embedContent / batch map
POST	/v1beta/models/{m}:embedContent	Native Gemini embeddings
POST	/v1beta/models/{m}:batchEmbedContents	Native Gemini batch embeddings
POST	/v1/responses	OpenAI Responses (stream SSE supported)
GET / DELETE	/v1/responses/{id}	?provider= or default OpenAI dialect
POST / GET / DELETE	/v1/files…	OpenAI Files or Anthropic Files when anthropic-version is set
POST / GET / DELETE	/v1/vector_stores…	OpenAI Vector stores (upstream-owned)
POST / GET	/v1/uploads…	OpenAI Uploads (multipart resume; upstream-owned)
POST / GET / DELETE	/v1/containers…	OpenAI Containers (upstream-owned)
GET	/v1/files/{id}/content	Streamed download
POST / GET	/v1/messages/batches…	Anthropic Message Batches (kind: anthropic only)
POST / GET / DELETE	/v1/skills…	Anthropic Skills Management (beta headers forwarded)
POST / GET / DELETE	/v1/tunnels…	Anthropic MCP tunnels (beta headers forwarded)
POST / GET / DELETE	/v1/memory_stores…	Anthropic agent memory stores
POST / GET	/v1/batches…	OpenAI Batches (openai / openai_compat; cancel via POST …/cancel)
POST / GET	/v1/fine_tuning/jobs…	OpenAI Fine-tuning jobs, cancel, events, checkpoints
POST	/v1/moderations	OpenAI-family passthrough
POST	/v1/tokenizers/estimate-token-count	Moonshot helper (openai_compat; ?provider= / default)
GET	/v1/users/me/balance	Moonshot balance helper (openai_compat)
GET	/v1/credits · /v1/key · /v1/generation	OpenRouter account/generation helpers
POST / GET / DELETE	/v1/assistants… · /v1/threads…	Legacy Assistants v2 (prefer Responses; send OpenAI-Beta)

Files and batches are upstream-owned (no gateway disk store). Body cap: max_body_bytes (default 32 MiB).

Anthropic Files use the same /v1/files* paths as OpenAI; presence of anthropic-version selects the Anthropic path. Client anthropic-beta (including unknown values) is forwarded.

Media & audio

Method	Path	Notes
POST	/v1/images/generations · /edits · /variations	OpenAI-shaped
POST	/v1/images · /v1/images/edits	Anthropic-gateway when anthropic-version set
POST	/v1/videos · GET /v1/videos/{id} · /content	Video jobs
POST	/v1/audio/speech	TTS (OpenAI path; Anthropic-version → Anthropic-gateway contract)
POST	/v1/audio/transcriptions · /translations	STT multipart/JSON
POST	/v1beta/models/{m}:generateSpeech	Google-shaped TTS → Gemini AUDIO generateContent

openai_compat media/realtime defaults off — opt in with capabilities in YAML.

Realtime (WebSocket)

Path	Notes
GET /v1/realtime	OpenAI Realtime upgrade; requires capabilities.realtime
GET /v1beta/models/{m}:bidiGenerateContent	Google Live; kind: google + realtime capability

Same-protocol passthrough only. Cross-protocol Realtime↔Live attempts return unsupported_realtime_bridge. Session limits: realtime.max_sessions (default 1024), realtime.max_session_minutes (default 60).

TLS / wss: production upstreams work. Provider base_url may be https://… (or wss://…); the gateway dials TLS with system roots (TLS 1.2+) and HTTP Upgrade. Example:

providers:
  openai:
    kind: openai
    base_url: "https://api.openai.com/v1"   # dials wss via TLS
    api_key_env: OPENAI_API_KEY
    # capabilities.realtime defaults on for kind: openai

TCP keepalive (30s) is enabled on the upstream socket. Application WebSocket ping/pong frames from either peer are passed through raw. Auth uses the same modes as HTTP (api_key_env, oauth2, client_bearer, ADC/SA TokenSource).

Completions (experimental)

Method	Path	Notes
POST	/v1/completions	OpenAI-family Completions passthrough
POST	/beta/completions	Rewrites base …/v1 → …/beta (DeepSeek FIM)

Not multi-dialect translated. Prefer chat for normal use.

Conversations (not supported)

/v1/conversations* (including nested paths such as /{id}/items) returns HTTP 501 with an OpenAI-shaped error:

Field	Value
status	501
error.type / code	not_implemented
guidance	Prefer POST /v1/responses with client-side conversation/history state; use Files for durable assets

Decision: permanent skip of gateway-side conversation storage (stateless). Routes are registered so SDKs get a structured 501 instead of a bare 404. Do not add a gateway database or Redis thread store.

Alternatives:

1. POST /v1/responses (+ get/delete by id on upstream when supported)
2. Client-owned message history on subsequent chat/Responses calls
3. Files / vector-store workstreams for stored assets (upstream-owned)

Formal product decision (Option A permanent 501): docs/conversations-decision.md (#118).

Dialect pairing (chat)

Client dialect	Upstream	Path
OpenAI	openai / openai_compat	passthrough
OpenAI	anthropic / google	translated
Anthropic	anthropic	passthrough
Anthropic	openai / openai_compat / google	translated
Google	google	passthrough
Google	openai / openai_compat / anthropic	translated

Limits & timeouts

Limit	Default	Config / code
Request/response body	32 MiB	max_body_bytes (bytes); oversize → 413
HTTP ReadHeaderTimeout	10s	server
Upstream ResponseHeaderTimeout	60s	HTTP client
Idle conn	90s	HTTP client
count_tokens upstream	15s	request context
Realtime max sessions	1024	realtime.max_sessions
Realtime max duration	60 min	realtime.max_session_minutes
SIGTERM drain	30s	process
Webhook hook timeout	3s	hooks.webhook.timeout

---

Model routing

Public model resolves in order:

1. aliases — exact match (fast → deepseek/deepseek-chat)
2. provider/model — first segment is a configured provider name
3. Bare id — dialect default (defaults.openai_dialect / anthropic_dialect / google_dialect)

Missing default or unknown provider → 404 (dialect error envelope).

GET /v1/models is built from config only (no upstream fan-out). Each entry may include:

{
  "id": "fast",
  "object": "model",
  "created": 0,
  "owned_by": "llm-gateway",
  "capabilities": {
    "chat": true,
    "image_gen": false,
    "video_gen": false,
    "audio_speech": false,
    "audio_transcribe": false,
    "realtime": false
  }
}

Flags come from provider kind defaults + optional YAML capabilities (text maps to JSON chat).

---

Configuration

Single YAML file. Unknown fields are rejected.

Field	Required	Description
listen	no	Bind address; default :8787 (GATEWAY_LISTEN)
providers	yes	Map of name → provider (≥1)
providers.<n>.kind	yes	openai | openai_compat | anthropic | google
providers.<n>.base_url	yes	Origin with version prefix; trailing / trimmed
providers.<n>.api_key_env	no	Env var; when set & non-empty, replaces client key
providers.<n>.capabilities	no	Override modality flags; nil → kind defaults (openai_compat = text only)
providers.<n>.auth	no	api_key (default) | adc | service_account | oauth2 | client_bearer | bearer
providers.<n>.oauth	when auth: oauth2	Token URL + client/refresh credentials (see OAuth / token sources)
providers.<n>.service_account_file	no	GCP SA JSON path; auto TokenSource with auth: service_account / adc
defaults.openai_dialect	no	Bare models on OpenAI ingress
defaults.anthropic_dialect	no	Bare models on Anthropic ingress
defaults.google_dialect	no	Bare models on Gemini ingress
aliases	no	Public id → provider/upstream-model
max_body_bytes	no	Default 33554432 (32 MiB)
observe_dropped_fields	no	Default false. When true, translate responses set X-Gateway-Dropped-Fields (names only) and usage dropped_fields
health_checks.enabled	no	Default false. Enables GET /v1/health/providers upstream probes
health_checks.timeout	no	Per-provider probe timeout (default 2s)
edge_auth	no	Optional shared-secret gate (see Auth)
realtime.*	no	Session caps
hooks.jsonl / hooks.webhook	no	Usage sinks

Provider kinds

Kind	Typical base	Auth
openai	https://api.openai.com/v1	Authorization: Bearer
openai_compat	DeepSeek, xAI, Moonshot, OpenRouter, Gemini …/v1beta/openai, vLLM	Bearer
anthropic	https://api.anthropic.com/v1	x-api-key + anthropic-version
google	https://generativelanguage.googleapis.com/v1beta	x-goog-api-key (or Bearer via auth: adc)

---

Auth & keys

Upstream credentials

The gateway reads a client credential from:

1. Authorization: Bearer <key>, or
2. x-api-key: <key>, or
3. x-goog-api-key: <key>

…and forwards it using the provider’s scheme, unless a server-held mode replaces it:

Mode	Behavior
api_key (default)	Client key, or api_key_env when set and non-empty
bearer	Always Authorization: Bearer (client or api_key_env)
client_bearer	Always forward client Bearer; never replace with api_key_env (multi-tenant / user OAuth)
oauth2	Built-in OAuth2 TokenSource from oauth: block (client_credentials or refresh_token)
adc / service_account	Bearer from TokenSource: inject via SetTokenSource, or auto from service_account_file / GOOGLE_APPLICATION_CREDENTIALS

Usage events include key_hash (12 hex chars of SHA-256 of the upstream credential) — correlate without storing secrets.

OAuth / token sources

Server-held OAuth2 (no cloud SDK; form POST + optional SA JWT):

providers:
  openai:
    kind: openai
    base_url: "https://api.openai.com/v1"
    auth: oauth2
    oauth:
      token_url: "https://oauth.example.com/token"
      client_id_env: OPENAI_OAUTH_CLIENT_ID
      client_secret_env: OPENAI_OAUTH_CLIENT_SECRET
      # or refresh_token_env for refresh grant (auto when refresh is set)
      scopes: ["api"]

  vertex:
    kind: google
    base_url: "https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT/locations/us-central1/publishers/google"
    auth: service_account
    service_account_file: /secrets/vertex-sa.json   # auto JWT → access token

• Tokens are cached until expires_in (30s skew). Concurrent refresh is single-flight.
• Prefer *_env for secrets; inline client_id / client_secret / refresh_token are for tests only.
• Edge auth is orthogonal to upstream OAuth — do not put provider refresh tokens in edge_auth.keys.
• ToS: consumer subscription OAuth (e.g. ChatGPT/Claude consumer) may forbid multi-user products; use operator-held credentials and vendor docs.

Multi-tenant pattern: edge_auth + auth: client_bearer so each client’s OAuth access token is forwarded upstream.

See SECURITY.md, docs/oauth-token-sources.md, and docs/wif-recipes.md (WIF / token_file).

Optional edge auth

By default the gateway does not authenticate callers (trusted network / external auth). To require a shared secret:

edge_auth:
  enabled: true
  keys_env: GATEWAY_EDGE_KEYS   # comma-separated

When enabled, every route except GET /healthz requires a matching key. Missing/invalid → 401. Constant-time compare; keys never logged. With api_key_env on providers, clients only need the edge key.

See SECURITY.md.

Forwarded client headers

When present: HTTP-Referer, Referer, X-Title, OpenAI-Organization, OpenAI-Project, anthropic-beta, client anthropic-version.
anthropic-beta is not allowlisted — unknown / future beta strings are forwarded unchanged.

---

Provider notes

Full comments: gateway.example.yaml. Matrices: docs/compatibility-matrix.md, docs/sdk-compatibility-matrix.md.

Provider	Kind	Notes
OpenAI	openai	Chat, Responses, Files, Moderations, images, video, audio, Realtime
Anthropic	anthropic	Messages, count_tokens, Files (+ beta), Batches
Google native	google	generateContent, embeddings, models list, Live, speech
Gemini OpenAI-compat	openai_compat	…/v1beta/openai base; opt-in media capabilities
DeepSeek	openai_compat	Chat + experimental Completions/FIM (/v1 or /beta)
OpenRouter / xAI / Moonshot / Groq / Qwen / …	openai_compat	Passthrough; set capabilities for media/realtime
Z.AI / Zhipu (GLM)	openai_compat	Regional bases (intl vs CN) — docs/providers/zai.md; wrong region ⇒ auth fail
Qwen (DashScope)	openai_compat	Regional bases + compatible-mode path — docs/providers/qwen.md; aliases qwen-turbo / qwen-plus
xAI (Grok)	openai_compat	Chat + Responses; Imagine images need image_gen — docs/providers/xai.md; alias grok
Groq	openai_compat	STT-first split routing — docs/providers/groq-stt.md; audio_transcribe + alias whisper-fast
Moonshot / Kimi	openai_compat	API keys, K3, Kimi Code CLI, helpers — docs/providers/moonshot-kimi.md; aliases kimi / kimi-k3; regional base
Moonshot helpers	openai_compat	POST /v1/tokenizers/estimate-token-count, GET /v1/users/me/balance via ?provider= / default OpenAI dialect (regional base)
Vertex	google + auth: adc / service_account	Auto SA JWT from service_account_file, or inject TokenSource; no Google SDK bundled

---

Passthrough vs translation

Passthrough (same family)

Client dialect matches provider kind → near-verbatim proxy: model rewrite, auth, headers, one usage event. Highest fidelity.

Translation (cross family)

Client and upstream disagree → parse to canonical IR, rebuild wire, stream map. Chat fidelity includes tools, structured outputs, thinking/reasoning, sampling knobs, document/audio blocks where mapped. Some vendor-only fields are still dropped — see docs/deprecation-policy.md (passthrough never drops; translation drop lists + semver) and testdata/fixtures/chat_translate/drops/. Non-function OpenAI tools: error on translate, forward on passthrough — docs/tools-policy.md.

Prompt caching: Anthropic cache_control, OpenAI prompt_cache_*, and Google cachedContent are preserved within family (including Anthropic translate rebuild). Cross-family hops drop foreign cache directives. Optional caching.auto_breakpoints can inject Anthropic breakpoints on OpenAI/Google → Anthropic translate (default off) — docs/cache-control-policy.md.

Not multi-dialect translated: Completions/FIM, most media jobs (family passthrough + limited speech translate), Files, Batches.

---

Hooks & usage events

Exactly one UsageEvent per proxied chat, media, embeddings, audio, responses, files, batches create, or realtime session (including errors). Not emitted for count_tokens, Gemini :countTokens, models discovery, or healthz.

Sink	Config
JSONL	hooks.jsonl.output: stdout | stderr | file path
Webhook	hooks.webhook.url (+ optional timeout, default 3s)
Go	gateway.WithHook(...) in library mode

Metrics / Prometheus

GET /metrics is served by prometheus/client_golang (promhttp) on a per-server registry:

Metric	Type	Labels
llm_gateway_requests_total	counter	status
llm_gateway_tokens_in_total / _out_total	counter	—
llm_gateway_request_duration_seconds	histogram	status
Go + process collectors	—	standard

Open when edge auth is on (like /healthz). Prefer hooks for high-cardinality labels (model/provider) / full billing detail.

Provider health

/healthz is process liveness only. Optional GET /v1/health/providers probes configured upstreams when health_checks.enabled: true (timeouts; no key logging).

Event shape (JSON)

Typical fields: request_id, time, dialect_in, provider, model, upstream_model, modality, transport, token counts, optional media, stream, status, http_status, latency_ms, key_hash. See hooks package docs.

---

Claude Code

Point Claude Code at the gateway (Anthropic Messages ingress). Same-family Anthropic is passthrough; OpenAI / xAI aliases are translated.

export ANTHROPIC_BASE_URL=http://localhost:8787
export ANTHROPIC_API_KEY=sk-…   # or edge key when edge_auth is on
# optional: ANTHROPIC_MODEL=deepseek/deepseek-chat
claude

Multi-provider with subscription OAuth (ChatGPT Plus/Pro, Claude Pro/Max, SuperGrok — not API keys):

# one-time: install helpers from the binary (or source examples/shell/… from a checkout)
llm-gateway helpers install && eval "$(llm-gateway helpers source)"

llm-gateway auth login chatgpt   # and/or claude, grok
export KEY=local-dev
cc-gateway-up
cc-gpt                           # or cc-grok / cc-gpt-grok / cc-multi
# /model grok-4.5 | /model composer-2.5 | /model gpt | /model sonnet
cc-gateway-logs -f               # usage + HTTP access

From a git checkout you can still use KEY=local-dev ./examples/claude-code-multi.sh gpt+grok or source examples/shell/claude-code-helpers.sh.

API-key recipe (alternative): examples/configs/claude-code-multi.yaml.

Guide: docs/claude-code-multi.md. Checklist: docs/claude-code-checklist.md.

Same subscriptions in Claude Desktop, Codex, Cursor, …

One gateway backs any OpenAI- or Anthropic-compatible app.

From a release binary (no git checkout):

llm-gateway helpers install    # or: llm-gateway load-helpers
# add to ~/.zshrc the "source …" lines it prints, then:
source ~/.config/inja-gateway/shell/claude-code-helpers.sh
source ~/.config/inja-gateway/shell/cursor-helpers.sh
source ~/.config/inja-gateway/shell/apps-helpers.sh
export KEY=local-dev && cc-gateway-up

From a git checkout:

source examples/shell/claude-code-helpers.sh
export KEY=local-dev && cc-gateway-up
source examples/shell/apps-helpers.sh
apps-setup                    # Claude Desktop, Codex, Continue, Cline, Aider, …
apps-use-gateway              # write gateway configs; snapshot "default" for rollback
apps-use-default              # restore pre-gateway settings
# apps-status · apps-list-backups · apps-switch gateway|default

Templates: examples/apps/. Docs: Any app · Claude app · Codex.

---

Library use

import (
    gateway "github.com/inja-online/llm-gateway"
    "github.com/inja-online/llm-gateway/config"
    "github.com/inja-online/llm-gateway/hooks/jsonl"
)

cfg, err := config.Load("gateway.yaml")
// ...
hook, _ := jsonl.New(cfg.Hooks.JSONL.Output)
h, err := gateway.New(cfg, gateway.WithHook(hook))
// http.ListenAndServe(cfg.Listen, h)

Inject Vertex/ADC tokens with proxy.Server.SetTokenSource when embedding the package.

---

Architecture

cmd/gateway          → binary, flags, graceful shutdown
config/              → YAML, capabilities, edge_auth, body limit
proxy/               → HTTP/WS routing, passthrough, translation orchestration
canonical/           → dialect-neutral chat/image/video/audio/realtime types
ingress/{openai,anthropic,google}/
egress/{openai,anthropic,google}/
hooks/{jsonl,webhook}/

Invariant: same-family traffic prefers byte passthrough; translation only when dialects differ.

---

Deploy

Path	Use
Dockerfile	Distroless multi-stage build
docker-compose.yml	Local stack
deploy/k8s/gateway.yaml	Sample Deployment/Service
Release workflow	Tag v* → multi-arch binaries

Set secrets via env (api_key_env / keys_env); mount config read-only.

---

Development & CI

go test ./...
go test -race ./...
go test ./... -coverprofile=coverage.out && go tool cover -func=coverage.out | tail -1
go vet ./...
docker build -t llm-gateway:dev .

CI (push/PR): build, vet, go test -race -count=1 ./... (air-gapped httptest only — no -tags live), coverage ≥ 90%, binary smoke, Docker healthz.

Hermetic dialect anchors: docs/sdk-compatibility-matrix.md.

Docs site (Nimbus + Astro 7, GitHub Pages):

cd website && npm install && npm run dev   # http://localhost:4321/llm-gateway/

Product landing + docs tree under website/ · workflow: .github/workflows/docs.yml. Agent index: /llms.txt.

Release: git tag vX.Y.Z && git push origin vX.Y.Z

---

Roadmap

Shipped: multi-dialect chat fidelity, media/audio, Responses/Files/Batches, Realtime/Live passthrough, models capabilities, edge auth, AGPL-3.0.

Possible follow-ups:

• Optional full Realtime ↔ Live IR bridge (today: fail-closed)
• Deeper cross-dialect image/video generation translation
• Richer Prometheus histograms/labels beyond low-cardinality counters

---

Contributing

See CONTRIBUTING.md (modality checklist + layout).
Security: SECURITY.md · docs/security-multipart-review.md.
Changelog: CHANGELOG.md.

---

License

GNU Affero General Public License v3.0 (AGPL-3.0) © 2026 inja-online

This project is free software under the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

If you run a modified version of this software as a network service (for example a hosted LLM gateway), the AGPL requires that you offer the corresponding source code of that modified version to users of the service. See LICENSE for the full terms.