ToolAI.io · Chaîne GitHub
Un index factuel des projets de développement LLM, agent, MCP, RAG et AI, avec pour chaque entrée la licence, la configuration, le téléchargement, des captures d’écran et les ressources associées.
Données du dépôt public
xllm-ai/xllm
xLLM is a C++ inference framework for LLM, VLM, DiT and REC models. It targets high-throughput, low-latency deployment on several AI accelerator families and separates service-layer scheduling and availability from engine-layer computation.
NVIDIA/TensorRT
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
avifenesh/memra
A from-scratch LLM inference engine built in Rust and CUDA, specifically optimized for RTX 5090 (sm_120a) and H100 (sm_90a) architectures, delivering exactness-gated performance without relying on frameworks like ggml.
PaddlePaddle/Paddle-Lite
PaddlePaddle High Performance Deep Learning Inference Engine for Mobile and Edge (飞桨高性能深度学习端侧推理引擎）
ollama/ollama
A tool for running and managing large language models locally.
hal0ai/hal0
An open-source, self-hosted home AI inference platform designed to turn a Linux box into an OpenAI-compatible inference appliance, with native optimization for AMD Strix Halo hardware.
NVIDIA/Model-Optimizer
A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
mudler/LocalAI
A local inference engine for self-hosting models and AI services.
ray-project/ray
A distributed computing runtime for machine learning training, tuning and model serving.
MoonshotAI/checkpoint-engine
Checkpoint-engine is a simple middleware to update model weights in LLM inference engines
janhq/jan
A local AI chat application for running models on a personal computer.
Page 2 / 2 · 23 projets
