Open Source Linux MCP Gateways

Call all LLM APIs using the OpenAI format [Anthropic, Huggingface, Cohere, Azure OpenAI etc.] liteLLM supports streaming the model response back, pass stream=True to get a streaming iterator in response. Streaming is supported for OpenAI, Azure, Anthropic, and Huggingface models.

Bifrost is an LLM gateway designed to provide a unified OpenAI-compatible API front for many different model providers. It abstracts away the complexity of working directly with multiple backend providers (OpenAI, Anthropic, AWS Bedrock, Google Vertex, etc.), enabling you to plug in providers and switch between them without touching your client code. It is built to be high performance: in benchmark tests at 5,000 requests per second, it reportedly adds only microseconds of overhead and achieves perfect success rates with no failed requests. Bifrost supports features such as automatic fallback (failover between providers), load balancing across API keys/providers, and semantic caching to reduce latency and cost. It also includes observability with built-in metrics, tracing, logging, and supports governance features like rate limiting, access control, and cost budgeting. The architecture is modular: there is a core engine, plugin layers, and transport layers (HTTP APIs).

mcp-server-and-gw is a Model Context Protocol (MCP) gateway that bridges standard input/output to HTTP Server-Sent Events (SSE) transport. It includes an example MCP server and client, facilitating the development and testing of MCP implementations. This tool is particularly useful for integrating MCP servers with applications like Claude Desktop that may not natively support remote server connections. ​

Portkey AI Gateway aims to offer a blazing fast, secure, and flexible gateway for interacting with a wide variety of models and enforcing guardrails. It presents a single, friendly API through which you can route to 200+ LLMs, while applying configurable input/output guardrails to enforce policies or restrict certain content. It supports automatic retries, fallbacks, load balancing across providers or keys, and request timeouts to avoid latency spikes. The gateway is multimodal: it can handle text, vision, audio, and image models under a common interface. It also offers features for governance: role-based access, compliance with standards (SOC2, HIPAA, GDPR), secure key management, and logging/analytics of usage, latency, errors, and cost. The system integrates with agent frameworks like LangChain, Autogen, and others, enabling the building of more complex AI applications. It’s lightweight and optimized for low latency with a small footprint.

MCP Context Forge is a feature-rich gateway and registry that federates Model Context Protocol (MCP) servers and traditional REST services behind a single, governed endpoint. It exposes an MCP-compliant interface to clients while handling discovery, authentication, rate limiting, retries, and observability on the server side. The gateway scales horizontally, supports multi-cluster deployments on Kubernetes, and uses Redis for federation and caching across instances. Operators can define virtual servers, wire multiple transports, and optionally enable an admin UI for management and monitoring. Packaged for quick starts via PyPI and Docker, it targets production reliability with health checks, metrics, and structured logs. The project positions itself as an integration hub so agentic apps can “connect once, use many” backends with consistent policy and lifecycle control.

Unla is a lightweight, highly available MCP gateway written in Go that turns existing MCP servers or ordinary HTTP APIs into MCP-compliant services through configuration, not code changes. Its goal is to let teams “wire up” tools they already run—internal REST endpoints, third-party APIs, or local MCP servers—and present a single, reliable MCP interface to clients like Claude Desktop, Cursor, and IDEs. The gateway focuses on operational concerns you’d expect in production: multi-instance availability, health checking, and declarative routing that maps upstreams to MCP tools and resources. A quick-start and CLI make it easy to stand up an API server, while the package structure exposes helpers for people who want to embed or extend the gateway. Because it is itself MCP-speaking, Unla can sit in front of mixed fleets and normalize transports and schemas for clients. Documentation and pkg.go.dev pages reinforce the positioning as a stable, Go-native building block for MCP deployments.

Octelium is an open source, self-hosted unified secure-access platform built for modern infrastructure and hybrid environments. It positions itself as more than a typical VPN; it supports zero-trust network access (ZTNA), “BeyondCorp”-style access, API/AI gateway functionality, and even serves as a PaaS-like deployment surface. One of its key strengths is identity-based, application-layer (L7) aware control, meaning access decisions are made per request, with context and policy rather than simple network-level allow/block rules. It supports both client-based (e.g., WireGuard/QUIC tunnels) and client-less access models, which makes it flexible for both human users and automated workloads. The project also highlights self-hosted, no hidden “server-side” locked components, giving organizations greater ownership and control over access, rather than relying on proprietary SaaS.

Agent Stack is an open infrastructure platform designed to take AI agents from prototype to production, no matter how they were built. It includes a runtime environment, multi-tenant web UI, catalog of agents, and deployment flow that seeks to remove vendor lock-in and provide greater autonomy. Under the hood it’s built on the “Agent2Agent” (A2A) protocol, enabling interoperability between different agent ecosystems, runtime services, and frameworks. The platform supports agents built in frameworks like LangChain, CrewAI, etc., enabling them to be hosted, managed and shared through a unified interface. It also offers multi-model, multi-provider support (OpenAI, Anthropic, Gemini, IBM WatsonX, Ollama etc.), letting users compare performance and cost across models. For developers and organizations building AI-agent products or automations, Agent Stack gives a scaffold that handles the “plumbing”, so they can focus on logic and domain.

Kong is a next generation cloud-native API platform for multi-cloud and hybrid organizations. When building for the web, mobile, or Internet of Things, you’ll need a common functionality to run your software, and Kong is that solution. Kong acts as a gateway, connecting microservices requests and APIs natively while also providing load balancing, logging, monitoring, authentication, rate-limiting, and so much more through plugins. Kong is highly extensible as well as platform agnostic, connecting APIs across different environments, platforms and patterns. Achieve architectural freedom with Kong today.

MCPJungle is a self-hosted gateway and registry for the Model Context Protocol (MCP), aimed at managing tool/integration servers for AI agents within organizations. It offers a “single source of truth” registry where developers can register MCP servers and the tools they provide, and MCP clients (such as AI agents) discover and consume those tools through one gateway endpoint. This greatly simplifies the architecture when you have many MCP servers; agents only need to connect to one gateway rather than multiple endpoints. The platform supports enterprise-grade workflows; centralized tool management, access control, self-hosting so that internal servers and tools remain under your organization’s control, and registry metadata to track what tools exist and who can use them. For organizations building internal AI automation systems, MCPJungle helps enforce governance, tool discovery, and integration scalability.

MetaMCP is an all-in-one MCP aggregator, orchestrator, and middleware that merges many MCP servers into a single, policy-aware server you can point any MCP client at. It’s built to centralize discovery, apply middlewares (e.g., auth, rate limits, transforms), and present a unified catalog of tools and resources from diverse backends. The repository and site describe it as “the one MCP to manage all your MCPs,” with diagrams and examples showing how to compose fleets behind a single endpoint. It ships Dockerized for quick deployment and emphasizes dynamic aggregation so teams can register or remove servers without restarting clients. The org maintains related repos and a GUI app for cloud and self-hosted setups, with a note that the cloud demo is outdated while the open-source v2 evolves. Overall, MetaMCP aims to simplify multi-server MCP operations for individuals and organizations.