Inference, Deployment & Runtime
llmgateway
theopenco/llmgateway
Route, manage, and analyze your LLM requests across multiple providers with a unified API interface.
★ 1.6K⑂ 172TypeScript
ToolAI.io · GitHub Channel
Trending AI Projects on GitHub
Track active open-source AI repositories across LLM, agent, MCP, RAG and coding topics, with stars and verified project facts.
Public repository data
Project index
Inference, Deployment & Runtime
vllm
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
★ 89.5K⑂ 21KPython
Agents & Multi-Agent
1flowbase
taichuy/1flowbase
An open-source AI gateway that allows local agent clients to publish fusion-style multi-model workflows as OpenAI and Claude-compatible virtual models, providing full observability into traces, tokens, latency, and costs.
★ 259⑂ 19Rust
Inference, Deployment & Runtime
TensorRT-LLM — Optimized LLM inference
NVIDIA/TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way
★ 14.4K⑂ 2.7KC++
Inference, Deployment & Runtime
hal0
hal0ai/hal0
An open-source, self-hosted home AI inference platform designed to turn a Linux box into an OpenAI-compatible inference appliance, with native optimization for AMD Strix Halo hardware.
★ 68⑂ 7Python
Agents & Multi-Agent
rig
0xplaygrounds/rig
⚙️🦀 Build modular and scalable LLM Applications in Rust
★ 8.3K⑂ 937Rust
MCP & Tool Calling
Askimo
askimo-ai/askimo
A native desktop AI client for chat, local RAG, multi-step AI workflows (Plans), and agent skills, supporting multiple cloud and local LLM providers while keeping user files strictly on the machine.
★ 347⑂ 73Kotlin
Agents & Multi-Agent
second-brain
henrydaum/second-brain
Second Brain is an agentic framework that acts as an operating system, using local file intelligence, workflow automation, and LLMs to complete tasks and communicate over multiple modalities and messaging platforms.
★ 607⑂ 75Python
MCP & Tool Calling
awesome-agentic-ai-zh
wenyuchiou/awesome-agentic-ai-zh
A trilingual (繁中 / English / 简中) learning roadmap for agentic AI: from LLM basics to multi-agent systems, with 240+ curated resources and hands-on examples. 中文 AI agent 學習地圖。
★ 6.1K⑂ 814Python
Evaluation, Observability & Safety
garak — LLM vulnerability scanning
NVIDIA/garak
the LLM vulnerability scanner
★ 8.9K⑂ 1.2KPython
MCP & Tool Calling
SBproxy
soapbucket/sbproxy
An open-source, self-hosted Enterprise AI Gateway and LLM proxy written in Rust, providing an OpenAI-compatible API for over 60 providers and supporting local model hosting, MCP tool federation, and robust traffic governance.
★ 49⑂ 1Rust
Agents & Multi-Agent
OpenGeni
cloudgeni-ai/opengeni
An open, self-hostable agentic runtime for organizations that provides durable, replayable agent sessions, human approvals, governed credentials and memory, and flexible compute targets including managed sandboxes and enrolled user hardware.
★ 56⑂ 5TypeScript
Page 4 / 11 · 131 projects