LLM Gateway Features

Everything you need to unify your AI infrastructure. One endpoint, all providers, zero hassle.

🌐 Supported Providers

Connect to all major LLM providers through a single OpenAI-compatible endpoint.

🟢 OpenAI (GPT-4o, GPT-4, GPT-3.5)
🔵 Google Gemini (2.0 Flash, 1.5 Pro)
🟣 Anthropic Claude (Opus, Sonnet, Haiku)
⚫ xAI Grok (Grok-3, Grok-3 Fast)
🌙 Kimi Moonshot (8K, 32K, 128K)
⚡ Groq (Llama 3.3, Mixtral)

✨ Core Features

🎯 Auto Model Detection

Just specify the model name. Gateway automatically routes to the correct provider. moonshot-v1-8k → Kimi, gpt-4o → OpenAI, etc.

🔄 Cascading Failover

When a provider fails or hits rate limits, requests automatically failover to the next available provider. Zero downtime.

🔑 Key Pooling

Add multiple API keys per provider. Gateway round-robins across healthy keys, maximizing your rate limits.

📊 Real-Time Analytics

Track requests, tokens, costs, and errors in real-time. Per-minute and per-hour metrics with historical data.

💰 Cost Optimization

Smart routing prefers cheaper providers for simple queries. Configure cost tiers and let the gateway optimize spend.

🛡️ Key Validation

API keys are validated against the provider before saving. No more debugging bad keys in production.

⚡ OpenAI Compatible

Drop-in replacement for OpenAI's API. Change your base URL and you're done. Works with any OpenAI SDK.

🎛️ Routing Control

Use X-Routing header to prefer specific providers, set cost tiers, or require capabilities like vision or tools.

📈 Usage Tracking

TPM, RPM, TPH tracking with configurable rate limits. Know exactly how much you're using in real-time.

📋 Release Notes

  • v1.3.0 February 4, 2026
    • Auto model-to-provider detection New
    • Model registry with 30+ models across 6 providers
    • Model aliases (gpt4 → gpt-4o, claude → claude-3.5-sonnet)
    • Cascade router for intelligent failover
    • Key validation before saving New
    • Docker restart policy set to always
  • v1.2.0 February 4, 2026
    • Smart routing based on model capabilities New
    • Real-time TPM/RPM/TPH tracking via Redis
    • Request queue with wait/webhook/poll strategies
    • Cost-tier based routing (cheap → balanced → premium)
    • New endpoints: /api/stats/realtime, /api/stats/limits
  • v1.1.0 February 4, 2026
    • Scheduler system for automated health checks New
    • Key health monitoring and alerts
    • Error breakdown statistics
    • Admin scheduler management API
  • v1.0.0 February 4, 2026
    • Initial release 🎉
    • Multi-provider proxy (OpenAI, Gemini, xAI, Kimi, Anthropic, Groq)
    • API key management with encryption
    • Usage tracking and cost calculation
    • Beautiful dark-mode dashboard
    • X-Routing header for provider preferences

🚀 Ready to Get Started?

Try LLM Gateway free. Pay only for your API usage with zero markup.

Open Dashboard →