Ringkasan
The project can run as an HTTP gateway, be embedded through its Go SDK, or serve as a replacement endpoint for supported provider SDKs. A built-in web interface supports configuration, monitoring, and analytics. The repository describes both open-source capabilities and additional enterprise deployment features.
Fitur utama
- Single OpenAI-compatible interface for 23+ providers
- Automatic retries, provider or model fallbacks, and load balancing
- Text, image, audio, and streaming request support
- Semantic response caching based on similarity
- MCP gateway capabilities for external tool access
- Virtual keys, usage tracking, budgets, rate limits, and fine-grained access control
- Prometheus metrics, distributed tracing, logging, monitoring, and analytics
- Web, API-driven, and file-based configuration
- Extensible plugin architecture
- HTTP gateway, Go SDK, and replacement endpoints for supported SDKs
- Enterprise options described for clustering, adaptive load balancing, guardrails, private networking, OIDC provisioning, custom controls, and governance
Persyaratan, instalasi, dan mulai cepat
Penggunaan
Kompatibilitas model dan kasus penggunaan
The README states support for 1,000+ models across 23+ providers. Named providers include OpenAI, Anthropic, AWS Bedrock, Google Vertex, Azure, Cerebras, Cohere, Mistral, Ollama, and Groq. It also documents integrations for OpenAI, Anthropic, AWS Bedrock, Google GenAI, LiteLLM, and LangChain SDKs. A complete model list is not included in the supplied repository record.
Catatan lisensi dan risiko
The repository metadata identifies the license as Apache-2.0, and the README states that the project is licensed under the Apache 2.0 License. Review the repository LICENSE file before redistribution or deployment.
Rilis dan pemeliharaan
Not stated in the repository metadata.