inference runtime
tiktoken — Fast tokenization
openai/tiktoken
tiktoken is a fast BPE tokeniser for use with OpenAI's models
★ 18,8K⑂ 1,6KRust
ToolAI.io · GitHub Channel
Open-source AI Projects on GitHub
An objective, fact-checked index of LLM, agent, MCP, RAG and AI development projects. Each entry includes license, setup, download, screenshots and related resources, with human review before publication.
2 published projects
10 editorial categories
AI facts first
Public repository data
Project index
inference runtime
TensorRT-LLM — Optimized LLM inference
NVIDIA/TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way
★ 14,2K⑂ 2,6KC++