Inferensi, Deployment & Runtime

any-llm: A Unified Python Interface for LLM Providers

mozilla-ai/any-llm

any-llm is a Python SDK for communicating with multiple LLM providers through one interface. It supports switching providers and models with minimal code changes while using official provider SDKs when available.

Buka repositori resmi ↗ Beranda proyek ↗
★ 2,2KBintang
⑂ 211Fork
45Isu terbuka
PythonBahasa
Apache-2.0Lisensi
Q@project.QualityScoreSkor editorial

Ringkasan

The project provides direct completion functions for scripts and experiments, plus a reusable AnyLLM client for applications making multiple requests. The repository describes support for OpenAI, Anthropic, Azure / Microsoft Foundry, Mistral, Ollama, OpenAI-compatible endpoints, and additional providers listed in its documentation. It is framework-agnostic and does not require a proxy.

Fitur utama

  • Unified completion interface across multiple LLM providers
  • Separate provider and model parameters or combined provider:model syntax
  • Direct synchronous API functions for stateless requests
  • Reusable AnyLLM clients with connection pooling
  • Streaming and tool support
  • OpenAI-style Responses API support for providers that implement it
  • Provider-specific installation extras
  • Type hints and actionable error messages
  • Use of official provider SDKs when available
  • Framework-agnostic operation without a required proxy
  • Model listing when supported by the selected provider
  • Support for custom OpenAI-compatible endpoints

Persyaratan, instalasi, dan mulai cepat

Install the SDK with extras for the required providers, for example: pip install 'any-llm-sdk[openai]', pip install 'any-llm-sdk[mistral,ollama]', or pip install 'any-llm-sdk[all]'. Set the relevant provider environment variable, such as OPENAI_API_KEY, ANTHROPIC_API_KEY, or MISTRAL_API_KEY. API keys can alternatively be passed in code.

Penggunaan

Quick start: install pip install 'any-llm-sdk[mistral,ollama]', set MISTRAL_API_KEY, import completion from any_llm, and call completion(model='mistral-small-latest', provider='mistral', messages=[{'role': 'user', 'content': 'Hello!'}]). Read the generated text from response.choices[0].message.content. For repeated requests, create a client with AnyLLM.create('mistral', api_key='your-mistral-api-key') and call llm.completion. A combined model string such as mistral:mistral-small-latest is also supported.

Kompatibilitas model dan kasus penggunaan

The README explicitly names OpenAI, Anthropic, Azure / Microsoft Foundry, Mistral, and Ollama, plus custom OpenAI-compatible endpoints. Model identifiers are passed directly to providers. Exact provider and model coverage should be checked at https://docs.mozilla.ai/any-llm/providers/. OpenAI-style Responses API functionality is limited to providers that implement that API.

Catatan lisensi dan risiko

Licensed under Apache License 2.0 according to the repository metadata and README. License reference: https://api.github.com/licenses/apache-2.0.

Rilis dan pemeliharaan

No release notes or version history are included in the supplied repository record. The repository was created on 2025-07-14, and the supplied metadata reports its latest push as 2026-08-03.

OpenVINO

openvinotoolkit/openvino

★ 10,6KC++

GPUStack

gpustack/gpustack

★ 5,4KPython