Inférence, déploiement et exécution

any-llm: A Unified Python Interface for LLM Providers

mozilla-ai/any-llm

any-llm is a Python SDK for communicating with multiple LLM providers through one interface. It supports switching providers and models with minimal code changes while using official provider SDKs when available.

Ouvrir le dépôt officiel ↗ Page d'accueil du projet ↗
★ 2,2KÉtoiles
⑂ 211Forks
45Problèmes ouverts
PythonLangue
Apache-2.0Licence
Q@project.QualityScoreScore éditorial

Vue d’ensemble

The project provides direct completion functions for scripts and experiments, plus a reusable AnyLLM client for applications making multiple requests. The repository describes support for OpenAI, Anthropic, Azure / Microsoft Foundry, Mistral, Ollama, OpenAI-compatible endpoints, and additional providers listed in its documentation. It is framework-agnostic and does not require a proxy.

Fonctionnalités clés

  • Unified completion interface across multiple LLM providers
  • Separate provider and model parameters or combined provider:model syntax
  • Direct synchronous API functions for stateless requests
  • Reusable AnyLLM clients with connection pooling
  • Streaming and tool support
  • OpenAI-style Responses API support for providers that implement it
  • Provider-specific installation extras
  • Type hints and actionable error messages
  • Use of official provider SDKs when available
  • Framework-agnostic operation without a required proxy
  • Model listing when supported by the selected provider
  • Support for custom OpenAI-compatible endpoints

Prérequis, installation et démarrage rapide

Install the SDK with extras for the required providers, for example: pip install 'any-llm-sdk[openai]', pip install 'any-llm-sdk[mistral,ollama]', or pip install 'any-llm-sdk[all]'. Set the relevant provider environment variable, such as OPENAI_API_KEY, ANTHROPIC_API_KEY, or MISTRAL_API_KEY. API keys can alternatively be passed in code.

Utilisation

Quick start: install pip install 'any-llm-sdk[mistral,ollama]', set MISTRAL_API_KEY, import completion from any_llm, and call completion(model='mistral-small-latest', provider='mistral', messages=[{'role': 'user', 'content': 'Hello!'}]). Read the generated text from response.choices[0].message.content. For repeated requests, create a client with AnyLLM.create('mistral', api_key='your-mistral-api-key') and call llm.completion. A combined model string such as mistral:mistral-small-latest is also supported.

Compatibilité des modèles et cas d’usage

The README explicitly names OpenAI, Anthropic, Azure / Microsoft Foundry, Mistral, and Ollama, plus custom OpenAI-compatible endpoints. Model identifiers are passed directly to providers. Exact provider and model coverage should be checked at https://docs.mozilla.ai/any-llm/providers/. OpenAI-style Responses API functionality is limited to providers that implement that API.

Licence et notes sur les risques

Licensed under Apache License 2.0 according to the repository metadata and README. License reference: https://api.github.com/licenses/apache-2.0.

Publication et maintenance

No release notes or version history are included in the supplied repository record. The repository was created on 2025-07-14, and the supplied metadata reports its latest push as 2026-08-03.

OpenVINO

openvinotoolkit/openvino

★ 10,6KC++

GPUStack

gpustack/gpustack

★ 5,4KPython