انفرنس، ڈپلائمنٹ اور رَن ٹائم
OpenVINO
openvinotoolkit/openvino
Open-source toolkit for optimizing and deploying AI inference across edge-to-cloud environments.
★ 10.6K⑂ 3.3KC++
ToolAI.io · GitHub چینل
GitHub پر ٹرینڈنگ AI پروجیکٹس
LLM، ایجنٹ، MCP، RAG اور کوڈنگ کے موضوعات میں فعال اوپن سورس AI ریپوزٹریز کو اسٹارز اور تصدیق شدہ پروجیکٹ معلومات کے ساتھ ٹریک کریں۔
عوامی ریپوزٹری کا ڈیٹا
پروجیکٹ انڈیکس
انفرنس، ڈپلائمنٹ اور رَن ٹائم
Mooncake: KVCache-Centric Infrastructure for Distributed LLM Serving
kvcache-ai/mooncake
Mooncake is a C++ infrastructure project for large-scale LLM inference and training. It separates prefill, decode, and storage resources while providing high-performance transfer, distributed KV-cache storage, and fault-tolerant expert-parallel communication.
★ 6.1K⑂ 1KC++
انفرنس، ڈپلائمنٹ اور رَن ٹائم
llm-d: Distributed LLM Inference on Kubernetes
llm-d/llm-d
llm-d is an open-source serving stack that adds distributed orchestration, routing, cache management, autoscaling, and batch processing around model servers such as vLLM and SGLang. It targets high-scale production inference on Kubernetes and modern hardware accelerators.
★ 4K⑂ 649Shell
انفرنس، ڈپلائمنٹ اور رَن ٹائم
GPUStack
gpustack/gpustack
GPUStack is an open-source GPU cluster manager for serving AI models and provisioning on-demand, SSH-accessible GPU instances. It orchestrates inference engines including vLLM, SGLang, and TensorRT-LLM across on-premises, Kubernetes, and cloud environments.
★ 5.4K⑂ 604Python
انفرنس، ڈپلائمنٹ اور رَن ٹائم
Model-Optimizer
NVIDIA/Model-Optimizer
A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
★ 3.3K⑂ 513Python
LLM اور فاؤنڈیشن ماڈلز
gpt-oss — Open-weight language model tooling
openai/gpt-oss
gpt-oss-120b and gpt-oss-20b are two open-weight language models by OpenAI
★ 20.3K⑂ 2.1KPython
انفرنس، ڈپلائمنٹ اور رَن ٹائم
FastDeploy
PaddlePaddle/FastDeploy
High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle
★ 3.7K⑂ 755Python
انفرنس، ڈپلائمنٹ اور رَن ٹائم
TensorRT
NVIDIA/TensorRT
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
★ 13.2K⑂ 2.4KC++
انفرنس، ڈپلائمنٹ اور رَن ٹائم
checkpoint-engine
MoonshotAI/checkpoint-engine
Checkpoint-engine is a simple middleware to update model weights in LLM inference engines
★ 982⑂ 100Python
کمپیوٹر وژن
sam2
facebookresearch/sam2
The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.
★ 19.6K⑂ 2.5KJupyter Notebook
انفرنس، ڈپلائمنٹ اور رَن ٹائم
Paddle-Lite
PaddlePaddle/Paddle-Lite
PaddlePaddle High Performance Deep Learning Inference Engine for Mobile and Edge (飞桨高性能深度学习端侧推理引擎）
★ 7.3K⑂ 1.6KC++
روبوٹکس اور ایج AI
edgeai-for-beginners
microsoft/edgeai-for-beginners
This course is designed to guide beginners through the exciting world of Edge AI, covering fundamental concepts, popular models, inference techniques, device-specific applications, model optimization, and the development of intelligent Edge AI agents.
★ 1.6K⑂ 362Jupyter Notebook
صفحہ 3 / 4 · 41 پروجیکٹس