ToolAI.io · GitHub 频道

推理、部署与运行时 · GitHub 上的开源 AI 项目

一个基于事实的 LLM、智能体、MCP、RAG 和 AI 开发项目索引;每个条目均包含许可证、设置、下载、截图及相关资源。

公开仓库数据

项目索引

2 项目
推理、部署与运行时

Model-Optimizer

NVIDIA/Model-Optimizer

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

★ 3.3K⑂ 513Python
Apache-2.0Q86
推理、部署与运行时

DeepInfer

deepseek-ai/DeepInfer

DeepSeek 自家线上服务用的高性能推理框架,开源版支持在 NVIDIA SM100 GPU 和华为昇腾 950 上离线运行 DeepSeek-V4.1-Flash。

★ 25.7K⑂ 3.2KPython
Apache-2.0Q82

最近更新

Craft Anythingstorytold/craft-anything★ 27.4K elephant-goldfish-modelgoogle/elephant-goldfish-model★ 15.6K DeepInferdeepseek-ai/DeepInfer★ 25.7K ComfyUIComfy-Org/ComfyUI★ 136.7K CrewAICrewAIInc/crewAI★ 59.5K Microsoft Agent Frameworkmicrosoft/agent-framework★ 14K

星标最多

ECCaffaan-m/ECC★ 248.8K Hermes Agentnousresearch/hermes-agent★ 227.1K tensorflowtensorflow/tensorflow★ 198.8K AutoGPTSignificant-Gravitas/AutoGPT★ 187.1K ollamaollama/ollama★ 180.2K markitdown:Document conversion and extractionmicrosoft/markitdown★ 174.9K