Resumen
Mooncake is the serving platform used for Kimi, Moonshot AI's LLM service. Its disaggregated architecture uses available CPU, DRAM, SSD/NVMe, network, and accelerator resources to create a shared KV-cache pool. The main components are Transfer Engine, Mooncake Store, Mooncake EP, and Mooncake Process Group. The repository reports integration with serving and distributed-computing ecosystems including SGLang, vLLM, TensorRT-LLM, LMCache, LMDeploy, NIXL, and PyTorch.
Características principales
- Unified batched data movement across heterogeneous storage, networks, and accelerators
- Topology-aware routing, multi-NIC bandwidth aggregation, and automatic transfer failover
- Distributed KV-cache and model-weight management through Mooncake Store
- DRAM and SSD/NVMe cache tiers with object replication, placement, pinning, and eviction controls
- Disaggregated prefill and decode serving
- Cross-instance KV-cache sharing and reuse
- Fault-tolerant expert-parallel dispatch and combine operations
- PyTorch ProcessGroup integration with failure reporting and rank recovery
- Integrations described for SGLang, vLLM, TensorRT-LLM, LMCache, LMDeploy, NIXL, and related systems
- Support described for TCP, RDMA, AWS EFA, NVMe-oF, NVLink, HIP, Barex, CXL, and Ascend-family transports
Requisitos, instalación y guía rápida
Uso
Compatibilidad de modelos y casos de uso
Mooncake is infrastructure rather than a model runtime tied to one model family. The supplied data mentions use with Kimi, Kimi-K2, DeepSeek deployments, GLM 5.2 FP8, LLaMA3-70B benchmark sizing, large mixture-of-experts systems, and multimodal pipelines. This does not establish universal compatibility with every model. Exact supported model lists and version constraints are not stated in the repository metadata.
Notas sobre la licencia y los riesgos
The repository metadata identifies the license as Apache-2.0 and provides https://api.github.com/licenses/apache-2.0. Users should review the repository's license file and the licenses of integrated runtimes, drivers, transports, and serving frameworks before deployment.
Lanzamiento y mantenimiento
The supplied update history reports that Transfer Engine was open-sourced on November 28, 2024 and Mooncake Store on March 7, 2025. It also records integrations or adoption involving vLLM, SGLang, TensorRT-LLM, LMCache, LMDeploy, NIXL, vLLM-Ascend, TorchSpec, and other projects. The repository metadata reports 6,132 stars, 1,047 forks, 462 open issues, a creation timestamp of 2024-06-25T05:21:05Z, and that the repository is not archived. No formal latest release version is stated.