Overzicht
The project can run as an HTTP gateway, be embedded through its Go SDK, or serve as a replacement endpoint for supported provider SDKs. A built-in web interface supports configuration, monitoring, and analytics. The repository describes both open-source capabilities and additional enterprise deployment features.
Belangrijkste functies
- Single OpenAI-compatible interface for 23+ providers
- Automatic retries, provider or model fallbacks, and load balancing
- Text, image, audio, and streaming request support
- Semantic response caching based on similarity
- MCP gateway capabilities for external tool access
- Virtual keys, usage tracking, budgets, rate limits, and fine-grained access control
- Prometheus metrics, distributed tracing, logging, monitoring, and analytics
- Web, API-driven, and file-based configuration
- Extensible plugin architecture
- HTTP gateway, Go SDK, and replacement endpoints for supported SDKs
- Enterprise options described for clustering, adaptive load balancing, guardrails, private networking, OIDC provisioning, custom controls, and governance
Vereisten, installatie en snel aan de slag
Gebruik
Modelcompatibiliteit en gebruiksscenario's
The README states support for 1,000+ models across 23+ providers. Named providers include OpenAI, Anthropic, AWS Bedrock, Google Vertex, Azure, Cerebras, Cohere, Mistral, Ollama, and Groq. It also documents integrations for OpenAI, Anthropic, AWS Bedrock, Google GenAI, LiteLLM, and LangChain SDKs. A complete model list is not included in the supplied repository record.
Licentie- en risiconotities
The repository metadata identifies the license as Apache-2.0, and the README states that the project is licensed under the Apache 2.0 License. Review the repository LICENSE file before redistribution or deployment.
Release en onderhoud
Not stated in the repository metadata.