Apache APISIX
No vendor lock-in
Open-source data plane, open plugin model, and portable configuration for APIs and AI workloads.
Apache APISIX AI Gateway
Open-Source AI Gateway for LLMs and AI Agents
Use Apache APISIX as an open-source LLM gateway and proxy for model routing, load balancing, retries, fallback, token rate limiting, security, and observability.
- 20+
- model providers
- 100+
- gateway plugins
- 0
- vendor lock-in
One gateway, two workloads
Manage API and AI traffic together
Keep the routing, security, observability, and operations model your teams already use while adding LLM traffic.
100+
Gateway capabilities
Authentication, traffic control, observability, serverless, and AI plugins.
Open AI plugin ecosystem
Load balancing, fallback, token controls, RAG, prompt policies, moderation, and auditing are all available as open-source plugins.
Architecture
A control point between applications and models
Built for production AI traffic
Reliability, control, and visibility at the gateway
Multi-LLM load balancing
Route across OpenAI, DeepSeek, Claude, Mistral, Gemini, and other providers with health checks and weighted balancing.
Token rate limiting
Control token consumption by Route, Service, Consumer, Consumer Group, or custom attributes in standalone and cluster deployments.
AI RAG
Connect enterprise knowledge to model requests at the gateway layer for grounded, context-aware responses.
Token observability
Track token usage through access logs and existing observability tools to control abuse and unexpected cost.
Retry and fallback
Use health checks, automatic retries, and fallback providers to keep AI applications available when an upstream model fails.
Prompt security
Apply prompt guards, decorators, templates, content moderation, logging, and auditing before traffic reaches a model.
Provider choice
Route to the model that fits each request
Use OpenAI, DeepSeek, Claude, Mistral, Gemini, self-hosted models, or a mix of providers without changing the application-facing gateway.
See the AI Gateway capabilities