Inference, Deployment & Runtime

any-llm: A Unified Python Interface for LLM Providers

mozilla-ai/any-llm

any-llm is a Python SDK for communicating with multiple LLM providers through one interface. It supports switching providers and models with minimal code changes while using official provider SDKs when available.

Open official repository ↗ Project homepage ↗
★ 2.2KStars
⑂ 211Forks
45Open issues
PythonLanguage
Apache-2.0License
Q@project.QualityScoreEditorial score

Overview

The project provides direct completion functions for scripts and experiments, plus a reusable AnyLLM client for applications making multiple requests. The repository describes support for OpenAI, Anthropic, Azure / Microsoft Foundry, Mistral, Ollama, OpenAI-compatible endpoints, and additional providers listed in its documentation. It is framework-agnostic and does not require a proxy.

Key features

  • Unified completion interface across multiple LLM providers
  • Separate provider and model parameters or combined provider:model syntax
  • Direct synchronous API functions for stateless requests
  • Reusable AnyLLM clients with connection pooling
  • Streaming and tool support
  • OpenAI-style Responses API support for providers that implement it
  • Provider-specific installation extras
  • Type hints and actionable error messages
  • Use of official provider SDKs when available
  • Framework-agnostic operation without a required proxy
  • Model listing when supported by the selected provider
  • Support for custom OpenAI-compatible endpoints

Requirements, installation and quick start

Install the SDK with extras for the required providers, for example: pip install 'any-llm-sdk[openai]', pip install 'any-llm-sdk[mistral,ollama]', or pip install 'any-llm-sdk[all]'. Set the relevant provider environment variable, such as OPENAI_API_KEY, ANTHROPIC_API_KEY, or MISTRAL_API_KEY. API keys can alternatively be passed in code.

Usage

Quick start: install pip install 'any-llm-sdk[mistral,ollama]', set MISTRAL_API_KEY, import completion from any_llm, and call completion(model='mistral-small-latest', provider='mistral', messages=[{'role': 'user', 'content': 'Hello!'}]). Read the generated text from response.choices[0].message.content. For repeated requests, create a client with AnyLLM.create('mistral', api_key='your-mistral-api-key') and call llm.completion. A combined model string such as mistral:mistral-small-latest is also supported.

Model compatibility and use cases

The README explicitly names OpenAI, Anthropic, Azure / Microsoft Foundry, Mistral, and Ollama, plus custom OpenAI-compatible endpoints. Model identifiers are passed directly to providers. Exact provider and model coverage should be checked at https://docs.mozilla.ai/any-llm/providers/. OpenAI-style Responses API functionality is limited to providers that implement that API.

License and risk notes

Licensed under Apache License 2.0 according to the repository metadata and README. License reference: https://api.github.com/licenses/apache-2.0.

Release and maintenance

No release notes or version history are included in the supplied repository record. The repository was created on 2025-07-14, and the supplied metadata reports its latest push as 2026-08-03.

OpenVINO

openvinotoolkit/openvino

★ 10.6KC++

GPUStack

gpustack/gpustack

★ 5.4KPython