inference runtime
tiktoken：Fast tokenization
openai/tiktoken
ToolAI 对 openai/tiktoken 的客观项目介绍。tiktoken is a fast BPE tokeniser for use with OpenAI's models
★ 18.8K⑂ 1.6KRust
ToolAI.io · GitHub Channel
GitHub 开源 AI 项目精选
以公开仓库事实为基础，持续发现 LLM、Agent、MCP、RAG 和 AI 开发工具。每个项目提供许可证、安装、下载、截图与关联内容，发布前保留人工审核。
2 已发布项目
10 固定分类
AI 项目事实优先
公开仓库资料
项目列表
inference runtime
TensorRT-LLM：Optimized LLM inference
NVIDIA/TensorRT-LLM
ToolAI 对 NVIDIA/TensorRT-LLM 的客观项目介绍。TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way
★ 14.2K⑂ 2.6KC++