Skillget

awesome-ai-gateways

mcp
Security Audit
Pass
Health Pass
  • License — License: CC0-1.0
  • Description — Repository has a description
  • Active repo — Last push 0 days ago
  • Community trust — 19 GitHub stars
Code Pass
  • Code scan — Scanned 6 files during light audit, no dangerous patterns found
Permissions Pass
  • Permissions — No dangerous permissions requested

No AI report is available for this listing yet.

SUMMARY

Awesome AI gateways: open source and commercial LLM gateways, MCP gateways, agent (A2A) gateways, model routers, specs, and guides.

README.md

Awesome AI Gateways Awesome

AI gateways sit between applications and the models, MCP tools, and agents they call, and apply routing, access control, spend limits, guardrails, and logging to that traffic.

Open source and commercial gateways for LLM, MCP, and agent-to-agent (A2A) traffic, the specifications they implement, and guides for choosing one. Entries are sorted alphabetically within each section.

Each software entry ends with its license and, for open source projects, its main implementation language. Proprietary marks commercial products and AI features that require a paid edition.

Contents

AI Gateways

Gateways built for model traffic: one API in front of many providers, with routing, failover, keys, budgets, and logging.

  • Agent Router - Gateway built on Envoy Gateway that gives applications one OpenAI-compatible API for hosted models, self-hosted inference, and MCP servers, formerly Envoy AI Gateway and now an Agentic AI Foundation project (Apache-2.0, Go).
  • AISIX - Single static binary from the creators of Apache APISIX that puts one OpenAI-compatible API in front of LLM providers and routes, secures, caches, and observes LLM and agent traffic (Apache-2.0, Rust).
  • Bifrost - Single-binary gateway for LLM, MCP, and agent traffic behind one OpenAI-compatible API, with failover, load balancing, semantic caching, virtual keys, budgets, air-gapped deployment, and a published overhead of about 11 microseconds per request at 5,000 RPS on one t3.xlarge (Apache-2.0, Go).
  • F5 AI Gateway - Control plane that combines a model gateway, an MCP gateway, and guardrails under one policy model for model, agent, and tool traffic (Proprietary).
  • GoModel - Lightweight gateway with OpenAI-compatible and Anthropic-compatible APIs across providers, response caching, spend tracking, and a dashboard for adding providers (MIT, Go).
  • Helicone - LLM observability platform with a built-in AI gateway that reaches 100+ models through the OpenAI API format with routing and automatic fallbacks, plus request logging and prompt management (Apache-2.0, TypeScript).
  • LiteLLM - Python SDK and proxy server that call 100+ LLM providers in the OpenAI format, with the proxy adding virtual keys, spend tracking, guardrails, load balancing, and an admin dashboard (MIT, Python).
  • LLM Gateway - Gateway and dashboard that route requests to multiple LLM providers through one API, keep provider keys in one place, and track token usage and cost (AGPL-3.0, TypeScript).
  • MLflow AI Gateway - Component of the MLflow platform that gives an organization one secure endpoint for multiple LLM providers such as OpenAI and Anthropic (Apache-2.0, Python).
  • New API - Self-hosted gateway with a web console that exposes OpenAI-, Claude-, and Gemini-compatible APIs over many upstream providers and manages routing, access, usage, and cost per team (AGPL-3.0, Go).
  • Otari - OpenAI-compatible gateway from Mozilla.ai that routes one endpoint to 40+ providers, issues virtual keys, enforces budgets, and tracks usage (Apache-2.0, Python).
  • Plano - Out-of-process proxy and data plane for agentic apps that handles LLM routing with its own routing model, agent orchestration, guardrails, and tracing (Apache-2.0, Rust).
  • Portkey AI Gateway - Gateway that routes to 1,600+ models through one API with retries, fallbacks, load balancing, conditional routing, and guardrails, with an MCP gateway in Portkey's hosted platform (MIT, TypeScript).
  • Traceloop Hub - LLM gateway that exposes an OpenAI-compatible API over OpenAI, Anthropic, Azure OpenAI, Vertex AI, and Bedrock and records every call as OpenTelemetry traces (Apache-2.0, Rust).
  • TrueFoundry AI Gateway - Enterprise gateway for hundreds of models with unified governance and real-time observability, offered alongside TrueFoundry's MCP gateway and agent gateway (Proprietary).

MCP Gateways

Gateways that put many MCP servers behind one endpoint and add authentication, tool filtering, and audit logs. agentgateway, Bifrost, Kong Gateway, LiteLLM, and Portkey AI Gateway, listed in other sections, also handle MCP traffic.

  • Archestra - Platform with an MCP gateway using OAuth and on-behalf-of authentication, an LLM gateway, an A2A gateway, an MCP registry, SSO, RBAC, and guardrails (AGPL-3.0, TypeScript).
  • ContextForge - IBM registry and proxy that federates MCP servers, A2A agents, and REST or gRPC APIs into one endpoint with central governance, discovery, and observability (Apache-2.0, Python).
  • Docker MCP Gateway - Docker CLI plugin behind the MCP Toolkit in Docker Desktop that runs and deploys MCP servers from the Docker MCP Catalog and serves them to clients through one gateway (MIT, Go).
  • Gram - Open source stack behind Speakeasy's AI control plane, which manages company-wide access to MCP servers, skills, and assistants with fine-grained permissions, threat detection, and observability (AGPL-3.0, Go).
  • Kuadrant MCP Gateway - Envoy-based gateway that aggregates and routes multiple MCP servers behind one endpoint and uses Istio and Gateway API policy attachment for authentication, authorization, and rate limiting (Apache-2.0, Go).
  • Lunar.dev - API consumption gateway for applications and agents whose MCPX component aggregates multiple MCP servers into one gateway with shared policy and observability (MIT, TypeScript).
  • MCP Gateway & Registry - Governed entry point and registry for MCP servers, agents, and skills, with OAuth authentication and central discovery (Apache-2.0, Python).
  • MCPHub - Self-hosted MCP gateway and control plane that connects local and remote MCP servers once and serves them through stable endpoints with authentication and scoped credentials (Apache-2.0, TypeScript).
  • MCPJungle - Self-hosted MCP gateway that registers servers once and serves them to Claude, Cursor, Codex, or custom agents through a single endpoint with access control and observability (MPL-2.0, Go).
  • Microsoft MCP Gateway - Stateless reverse proxy and management layer for MCP servers on Kubernetes, with request routing, authorization, and server lifecycle management (MIT, C#).
  • Mintive - MCP gateway and LLM gateway, formerly MintMCP, that gives each agent its own identity and governs access to models and tools by role (Proprietary).
  • Obot - Self-hosted AI governance platform with MCP and LLM gateways, MCP and skills registries, and isolated hosting for MCP servers (MIT, Go).
  • Runlayer - AI control plane that routes MCP and LLM calls through one gateway and discovers agent, MCP, and skill activity across endpoints and clients (Proprietary).
  • ToolHive - Platform that runs each MCP server in an isolated container, enforces identity and access policy per request, and adds observability for MCP in production (Apache-2.0, Go).
  • Unla - Lightweight gateway that turns existing MCP servers and HTTP APIs into MCP endpoints through configuration, without code changes (MIT, Go).

Agent Gateways

Gateways for agent-to-agent (A2A) traffic and agent egress. Archestra, ContextForge, Gravitee, Kong Gateway, and Azure API Management, listed in other sections, also proxy A2A traffic.

  • agentgateway - Linux Foundation proxy built on MCP and A2A that acts as an LLM gateway, an MCP gateway with tool federation and OAuth, and an A2A gateway for agent-to-agent communication (Apache-2.0, Rust).
  • Amazon Bedrock AgentCore Gateway - Managed AWS entry point for agent traffic that turns APIs and Lambda functions into MCP tools, fronts other agents including A2A traffic, and routes inference across model providers (Proprietary).
  • Google Cloud Agent Gateway - Enforcement point in the Gemini Enterprise Agent Platform that applies access policies, mTLS, agent identities, and MCP security to agent-to-agent and agent-to-tool traffic (Proprietary).
  • Solo Enterprise for agentgateway - Commercial agentgateway offering from Solo.io for LLM, MCP, and inference traffic, with centralized provider key storage, enterprise IAM integration, and token-based rate limits (Proprietary).

API Gateways with AI Features

General-purpose API gateways and API management platforms that added LLM, MCP, or A2A support.

  • Apache APISIX - API gateway whose AI plugins proxy requests to LLM providers, load balance across model instances, limit token consumption, and apply prompt controls (Apache-2.0, Lua).
  • APIPark - AI gateway and API developer portal that connects 100+ models behind a standard request format, turns prompts into APIs, and shares them across teams (Apache-2.0, Go).
  • Gravitee - API management platform whose AI Agent Management pack, part of the Enterprise Edition, adds LLM proxies, MCP proxies, A2A proxies, and an agent catalog (Proprietary).
  • Higress - CNCF sandbox API gateway built on Istio and Envoy, with WebAssembly plugins for proxying many model providers and hosting MCP servers (Apache-2.0, Go).
  • Kong Gateway - API gateway whose AI Gateway plugins proxy LLM traffic and govern MCP and A2A traffic, with some plugins, such as AI Proxy Advanced, limited to Kong's AI Gateway Enterprise offering (Apache-2.0, Lua).
  • KrakenD AI Gateway - Enterprise Edition features for routing LLM traffic, with prompt guards, AI budget control, semantic caching, monitoring, and an MCP gateway (Proprietary).
  • MuleSoft Agent Fabric - Salesforce MuleSoft product that applies one set of policies to agents, MCP servers, and models without rebuilding or instrumenting them (Proprietary).
  • Traefik Hub AI Gateway - Control layer on top of the Traefik Hub API Gateway that adds middlewares for chat completion and other LLM traffic, alongside a separate MCP gateway (Proprietary).
  • Tyk - API gateway with an MCP gateway for governing agent access to MCP servers, including servers generated from REST APIs, plus Tyk AI Studio for LLM access management (MPL-2.0, Go).
  • Zuplo AI Gateway - Proxy between apps and LLM providers in the Zuplo API platform, with per-app policy chains for cost control, model access, guardrails, and observability (Proprietary).

Cloud and Platform AI Gateways

AI gateways offered as part of a cloud, data, or developer platform.

  • Apigee - Google Cloud API management used as an AI gateway to govern and secure LLM APIs and give developers consistent access to models across clouds (Proprietary).
  • Azure API Management AI Gateway - Azure API Management policies for LLM APIs, MCP servers, and A2A agent APIs, including token rate limits, semantic caching, load balancing, and circuit breakers (Proprietary).
  • Cloudflare AI Gateway - Managed gateway on Cloudflare's network with analytics, caching, rate limiting, and model fallback for requests to many AI providers (Proprietary).
  • Databricks Unity Gateway - Governance layer built on Unity Catalog for traffic between models, agents, MCP servers, and tools, with routing, service policies, and usage monitoring (Proprietary).
  • Netlify AI Gateway - Netlify service that lets code deployed on Netlify call popular AI models without separate provider accounts, credit balances, or API keys (Proprietary).
  • Vercel AI Gateway - Managed gateway for calling models across providers from any infrastructure, with centralized credentials, request logs, spend budgets, routing, and provider failover (Proprietary).

Model Routers and Aggregators

Hosted APIs for many models behind one key, and routers that pick a model for each request.

  • Amazon Bedrock Intelligent Prompt Routing - Serverless Bedrock endpoint that routes each request between models in the same family based on predicted response quality (Proprietary).
  • Microsoft Foundry Model Router - Trained language model, deployed like any other Foundry model, that routes each prompt in real time to the most suitable underlying LLM (Proprietary).
  • Not Diamond - Model router for coding agents that predicts which model to use for each input and works with existing harnesses and gateways (Proprietary).
  • OpenRouter - Hosted API for hundreds of models behind one endpoint, with automatic fallbacks and cost-based provider selection (Proprietary).
  • Requesty - Hosted gateway and router for 600+ models through one OpenAI-compatible API, with routing, caching, failover, spend limits, and EU data residency (Proprietary).
  • vLLM Semantic Router - Policy-driven router from the vLLM project that selects or combines models for each call across local, private, and cloud backends (Apache-2.0, Go).

Inference Gateways

Gateways and routers for self-hosted model serving on Kubernetes, which route on model server state such as queue depth and prefix cache.

  • AIBrix - Kubernetes control plane for vLLM, SGLang, or TensorRT-LLM whose OpenAI-compatible gateway routes each request from live engine state (Apache-2.0, Go).
  • Gateway API Inference Extension - Kubernetes SIG project that uses Envoy external processing to turn a compatible Gateway API implementation into an inference gateway for self-hosted models (Apache-2.0, Go).
  • llm-d - CNCF sandbox distributed inference stack for Kubernetes with prefix-cache-aware and load-aware request routing (Apache-2.0, Python).
  • Shepherd Model Gateway - Engine-agnostic model-routing gateway that manages worker lifecycles and balances traffic across self-hosted engines such as vLLM, TensorRT-LLM, and SGLang and cloud providers (Apache-2.0, Rust).

Specifications and Standards

Guides

Vendor-neutral guides from the AI Gateways organization.

Articles

Related Lists

Contributing

Contributions are welcome, including from vendors who disclose their affiliation. Read the contribution guidelines before opening a pull request; an automated review checks each entry within a few minutes.

Reviews (0)

No results found