Inference, Deployment & Runtime
ray
ray-project/ray
A distributed computing runtime for machine learning training, tuning and model serving.
★ 43.7K⑂ 8KPython
ToolAI.io · GitHub Channel
Open-source AI Projects on GitHub
A fact-based index of LLM, agent, MCP, RAG and AI development projects, with license, setup, download, screenshots and related resources for each entry.
Public repository data
Project index
Multimodal AI
ComfyUI
Comfy-Org/ComfyUI
A node-based interface and execution engine for building image and other generative AI workflows.
★ 136.7K⑂ 16.3KPython
Multimodal AI
TRELLIS.2: image-to-3D assets with materials
microsoft/TRELLIS.2
Microsoft’s 3D generation project with image-to-3D inference, PBR texturing and GLB export for GPU-equipped asset workflows.
★ 11.2K⑂ 1.4KPython
Inference, Deployment & Runtime
NVIDIA Container Toolkit: NVIDIA GPU access for containers
NVIDIA/nvidia-container-toolkit
Configures NVIDIA GPU access for container runtimes such as Docker, supporting containerized inference, training and CUDA workloads.
★ 4.6K⑂ 589Go
Robotics & Edge AI
edgeai-for-beginners
microsoft/edgeai-for-beginners
This course is designed to guide beginners through the exciting world of Edge AI, covering fundamental concepts, popular models, inference techniques, device-specific applications, model optimization, and the development of intelligent Edge AI agents.
★ 1.6K⑂ 362Jupyter Notebook
Inference, Deployment & Runtime
checkpoint-engine
MoonshotAI/checkpoint-engine
Checkpoint-engine is a simple middleware to update model weights in LLM inference engines
★ 982⑂ 100Python
LLM & Foundation Models
Qwen3 — Open-weight multilingual language models
QwenLM/Qwen3
Qwen3 is the large language model series developed by Qwen team, Alibaba Cloud
★ 27.5K⑂ 2KPython
LLM & Foundation Models
Qwen3-Coder — Code-focused language models
QwenLM/Qwen3-Coder
Qwen3-Coder is the code version of Qwen3, the large language model series developed by Qwen team
★ 16.8K⑂ 1.2KPython
Inference, Deployment & Runtime
TensorRT-LLM — Optimized LLM inference
NVIDIA/TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way
★ 14.4K⑂ 2.7KC++
Inference, Deployment & Runtime
jan
janhq/jan
A local AI chat application for running models on a personal computer.
★ 44.3K⑂ 3KTypeScript
Computer Vision
sam2
facebookresearch/sam2
The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.
★ 19.6K⑂ 2.5KJupyter Notebook
Page 4 / 4 · 47 projects