ToolAI.io · Kanal GitHub
Indeks berbasis fakta untuk proyek pengembangan LLM, agen, MCP, RAG, dan AI, dengan lisensi, pengaturan, unduhan, tangkapan layar, dan sumber daya terkait untuk setiap entri.
Data repositori publik
xllm-ai/xllm
xLLM is a C++ inference framework for LLM, VLM, DiT and REC models. It targets high-throughput, low-latency deployment on several AI accelerator families and separates service-layer scheduling and availability from engine-layer computation.
NVIDIA/TensorRT
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
avifenesh/memra
A from-scratch LLM inference engine built in Rust and CUDA, specifically optimized for RTX 5090 (sm_120a) and H100 (sm_90a) architectures, delivering exactness-gated performance without relying on frameworks like ggml.
PaddlePaddle/Paddle-Lite
PaddlePaddle High Performance Deep Learning Inference Engine for Mobile and Edge (飞桨高性能深度学习端侧推理引擎）
ollama/ollama
A tool for running and managing large language models locally.
hal0ai/hal0
An open-source, self-hosted home AI inference platform designed to turn a Linux box into an OpenAI-compatible inference appliance, with native optimization for AMD Strix Halo hardware.
NVIDIA/Model-Optimizer
A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
mudler/LocalAI
A local inference engine for self-hosting models and AI services.
ray-project/ray
A distributed computing runtime for machine learning training, tuning and model serving.
MoonshotAI/checkpoint-engine
Checkpoint-engine is a simple middleware to update model weights in LLM inference engines
janhq/jan
A local AI chat application for running models on a personal computer.
Halaman 2 / 2 · 23 proyek
