MCP 与工具调用
berriai/litellm
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
★ 56.8K⑂ 10.8KPython
NOASSERTIONQ98
推理、部署与运行时
theopenco/llmgateway
Route, manage, and analyze your LLM requests across multiple providers with a unified API interface.
★ 1.6K⑂ 172TypeScript
NOASSERTIONQ98
推理、部署与运行时
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
★ 89.5K⑂ 21KPython
Apache-2.0Q98
智能体与多智能体
taichuy/1flowbase
一款开源 AI 网关,允许本地代理客户端将融合式多模型工作流发布为兼容 OpenAI 和 Claude 的虚拟模型,并提供对追踪、Token、延迟和成本的完全可观测性。
★ 259⑂ 19Rust
Apache-2.0Q89
推理、部署与运行时
NVIDIA/TensorRT-LLM
ToolAI 对 NVIDIA/TensorRT-LLM 的客观项目介绍。TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way
★ 14.4K⑂ 2.7KC++
NOASSERTIONQ82
推理、部署与运行时
hal0ai/hal0
一个开源、自托管的家庭AI推理平台,旨在将Linux设备转变为兼容OpenAI的推理设备,并针对AMD Strix Halo硬件进行了原生优化。
★ 68⑂ 7Python
Apache-2.0Q89
智能体与多智能体
0xplaygrounds/rig
⚙️🦀 Build modular and scalable LLM Applications in Rust
★ 8.3K⑂ 937Rust
MITQ98
MCP 与工具调用
askimo-ai/askimo
一款原生桌面 AI 客户端,支持聊天、本地 RAG、多步 AI 工作流(Plans)及代理技能,兼容多种云端和本地 LLM 提供商,同时确保用户文件严格保留在本地机器上。
★ 347⑂ 73Kotlin
AGPL-3.0Q94
智能体与多智能体
henrydaum/second-brain
Second Brain is an agentic framework that acts as an operating system, using local file intelligence, workflow automation, and LLMs to complete tasks and communicate over multiple modalities and messaging platforms.
★ 607⑂ 75Python
MITQ92
MCP 与工具调用
wenyuchiou/awesome-agentic-ai-zh
A trilingual (繁中 / English / 简中) learning roadmap for agentic AI: from LLM basics to multi-agent systems, with 240+ curated resources and hands-on examples. 中文 AI agent 學習地圖。
★ 6.1K⑂ 814Python
MITQ96
评估、可观测性与安全
NVIDIA/garak
ToolAI 对 NVIDIA/garak 的客观项目介绍。the LLM vulnerability scanner
★ 8.9K⑂ 1.2KPython
Apache-2.0Q84
MCP 与工具调用
soapbucket/sbproxy
一款使用 Rust 编写的开源、自托管企业级 AI 网关和 LLM 代理,提供兼容 OpenAI 的 API,支持 60 多家提供商及本地模型托管、MCP 工具联邦与强大的流量治理。
★ 49⑂ 1Rust
Apache-2.0Q86