Berri.ai

Main Features and Characteristics:

• Model Access: Supports access across 100+ LLMs, including Azure Access, Gemini Access, Bedrock Access, OpenAI Access, Anthropic Access, all in the OpenAI format.
• Cost Tracking: Accurately charge teams for their usage. Attribute cost to key/user/team/org; Automatic spend tracking across OpenAI/Azure/Bedrock/GCP/etc.; Tag-based spend tracking; Log spend to s3/gcs/etc.; Prompt formatting support for HF models.
• Budgets & Rate Limits: Provides Budgets & Rate Limits functionality.
• LLM Fallbacks: Provides LLM Fallbacks to ensure service availability.
• Other Features: Batches API, Guardrails, LLM Observability, Prompt Management, S3 Logging, Pass-Through Endpoints, Virtual Keys, Teams, Load Balancing, RPM/TPM limits.

Target Users: Platform teams, enterprises that need to give LLM access to a large number of developers and projects.

Core Advantages:

• Streamlines the complexities of transforming inputs and outputs across providers, saving months of work.
• Provides the latest LLM models to users usually within a day of them being released, avoiding hours of work each time a new model is announced.
• Standardizes logging, the OpenAI API, and authentication for all models, significantly reducing operational complexities and enabling quick adaptation to changing demands.
• Scale: 10M+ docker pulls, 1B+ requests served, 80% uptime, 425+ contributors.

Typical Use Cases: Netflix uses LiteLLM to give developers Day 0 LLM access; Lemonade uses LiteLLM and Langfuse to streamline the complexities of managing multiple LLM models; RocketMoney uses the LiteLLM proxy to streamline management of LLMs by standardizing logging, API, and authentication to swiftly adopt new models.

Pricing Information:

• Open Source: $0 Free. Includes 100+ LLM Provider Integrations, Langfuse/Arize Phoenix/Langsmith/OTEL Logging, Virtual Keys/Budgets/Teams, Load Balancing/RPM/TPM limits, LLM Guardrails.
• Enterprise: Get In Touch (Contact for pricing), Cloud or Self-Hosted. Includes Everything in OSS, Enterprise Support + Custom SLAs, JWT Auth, SSO, Audit Logs, and All Enterprise Features.