Suy luận, Triển khai & Thời gian chạy

any-llm: A Unified Python Interface for LLM Providers

mozilla-ai/any-llm

any-llm is a Python SDK for communicating with multiple LLM providers through one interface. It supports switching providers and models with minimal code changes while using official provider SDKs when available.

Mở kho lưu trữ chính thức ↗ Trang chủ dự án ↗
★ 2,2KSố sao
⑂ 211Fork
45Vấn đề đang mở
PythonNgôn ngữ
Apache-2.0Giấy phép
Q@project.QualityScoreĐiểm biên tập

Tổng quan

The project provides direct completion functions for scripts and experiments, plus a reusable AnyLLM client for applications making multiple requests. The repository describes support for OpenAI, Anthropic, Azure / Microsoft Foundry, Mistral, Ollama, OpenAI-compatible endpoints, and additional providers listed in its documentation. It is framework-agnostic and does not require a proxy.

Tính năng chính

  • Unified completion interface across multiple LLM providers
  • Separate provider and model parameters or combined provider:model syntax
  • Direct synchronous API functions for stateless requests
  • Reusable AnyLLM clients with connection pooling
  • Streaming and tool support
  • OpenAI-style Responses API support for providers that implement it
  • Provider-specific installation extras
  • Type hints and actionable error messages
  • Use of official provider SDKs when available
  • Framework-agnostic operation without a required proxy
  • Model listing when supported by the selected provider
  • Support for custom OpenAI-compatible endpoints

Yêu cầu, cài đặt và bắt đầu nhanh

Install the SDK with extras for the required providers, for example: pip install 'any-llm-sdk[openai]', pip install 'any-llm-sdk[mistral,ollama]', or pip install 'any-llm-sdk[all]'. Set the relevant provider environment variable, such as OPENAI_API_KEY, ANTHROPIC_API_KEY, or MISTRAL_API_KEY. API keys can alternatively be passed in code.

Cách sử dụng

Quick start: install pip install 'any-llm-sdk[mistral,ollama]', set MISTRAL_API_KEY, import completion from any_llm, and call completion(model='mistral-small-latest', provider='mistral', messages=[{'role': 'user', 'content': 'Hello!'}]). Read the generated text from response.choices[0].message.content. For repeated requests, create a client with AnyLLM.create('mistral', api_key='your-mistral-api-key') and call llm.completion. A combined model string such as mistral:mistral-small-latest is also supported.

Khả năng tương thích của mô hình và trường hợp sử dụng

The README explicitly names OpenAI, Anthropic, Azure / Microsoft Foundry, Mistral, and Ollama, plus custom OpenAI-compatible endpoints. Model identifiers are passed directly to providers. Exact provider and model coverage should be checked at https://docs.mozilla.ai/any-llm/providers/. OpenAI-style Responses API functionality is limited to providers that implement that API.

Ghi chú về giấy phép và rủi ro

Licensed under Apache License 2.0 according to the repository metadata and README. License reference: https://api.github.com/licenses/apache-2.0.

Phát hành và bảo trì

No release notes or version history are included in the supplied repository record. The repository was created on 2025-07-14, and the supplied metadata reports its latest push as 2026-08-03.

OpenVINO

openvinotoolkit/openvino

★ 10,6KC++

GPUStack

gpustack/gpustack

★ 5,4KPython