추론, 배포 및 런타임

any-llm: A Unified Python Interface for LLM Providers

mozilla-ai/any-llm

any-llm is a Python SDK for communicating with multiple LLM providers through one interface. It supports switching providers and models with minimal code changes while using official provider SDKs when available.

공식 저장소 열기 ↗ 프로젝트 홈페이지 ↗
★ 2.2K별점
⑂ 211포크 수
45미해결 이슈
Python언어
Apache-2.0라이선스
Q@project.QualityScore편집 점수

개요

The project provides direct completion functions for scripts and experiments, plus a reusable AnyLLM client for applications making multiple requests. The repository describes support for OpenAI, Anthropic, Azure / Microsoft Foundry, Mistral, Ollama, OpenAI-compatible endpoints, and additional providers listed in its documentation. It is framework-agnostic and does not require a proxy.

주요 기능

  • Unified completion interface across multiple LLM providers
  • Separate provider and model parameters or combined provider:model syntax
  • Direct synchronous API functions for stateless requests
  • Reusable AnyLLM clients with connection pooling
  • Streaming and tool support
  • OpenAI-style Responses API support for providers that implement it
  • Provider-specific installation extras
  • Type hints and actionable error messages
  • Use of official provider SDKs when available
  • Framework-agnostic operation without a required proxy
  • Model listing when supported by the selected provider
  • Support for custom OpenAI-compatible endpoints

요구 사항, 설치 및 빠른 시작

Install the SDK with extras for the required providers, for example: pip install 'any-llm-sdk[openai]', pip install 'any-llm-sdk[mistral,ollama]', or pip install 'any-llm-sdk[all]'. Set the relevant provider environment variable, such as OPENAI_API_KEY, ANTHROPIC_API_KEY, or MISTRAL_API_KEY. API keys can alternatively be passed in code.

사용 정보

Quick start: install pip install 'any-llm-sdk[mistral,ollama]', set MISTRAL_API_KEY, import completion from any_llm, and call completion(model='mistral-small-latest', provider='mistral', messages=[{'role': 'user', 'content': 'Hello!'}]). Read the generated text from response.choices[0].message.content. For repeated requests, create a client with AnyLLM.create('mistral', api_key='your-mistral-api-key') and call llm.completion. A combined model string such as mistral:mistral-small-latest is also supported.

모델 호환성 및 사용 사례

The README explicitly names OpenAI, Anthropic, Azure / Microsoft Foundry, Mistral, and Ollama, plus custom OpenAI-compatible endpoints. Model identifiers are passed directly to providers. Exact provider and model coverage should be checked at https://docs.mozilla.ai/any-llm/providers/. OpenAI-style Responses API functionality is limited to providers that implement that API.

라이선스 및 위험 참고 사항

Licensed under Apache License 2.0 according to the repository metadata and README. License reference: https://api.github.com/licenses/apache-2.0.

릴리스 및 유지 관리

No release notes or version history are included in the supplied repository record. The repository was created on 2025-07-14, and the supplied metadata reports its latest push as 2026-08-03.

OpenVINO

openvinotoolkit/openvino

★ 10.6KC++

GPUStack

gpustack/gpustack

★ 5.4KPython