개요
Mooncake is the serving platform used for Kimi, Moonshot AI's LLM service. Its disaggregated architecture uses available CPU, DRAM, SSD/NVMe, network, and accelerator resources to create a shared KV-cache pool. The main components are Transfer Engine, Mooncake Store, Mooncake EP, and Mooncake Process Group. The repository reports integration with serving and distributed-computing ecosystems including SGLang, vLLM, TensorRT-LLM, LMCache, LMDeploy, NIXL, and PyTorch.
주요 기능
- Unified batched data movement across heterogeneous storage, networks, and accelerators
- Topology-aware routing, multi-NIC bandwidth aggregation, and automatic transfer failover
- Distributed KV-cache and model-weight management through Mooncake Store
- DRAM and SSD/NVMe cache tiers with object replication, placement, pinning, and eviction controls
- Disaggregated prefill and decode serving
- Cross-instance KV-cache sharing and reuse
- Fault-tolerant expert-parallel dispatch and combine operations
- PyTorch ProcessGroup integration with failure reporting and rank recovery
- Integrations described for SGLang, vLLM, TensorRT-LLM, LMCache, LMDeploy, NIXL, and related systems
- Support described for TCP, RDMA, AWS EFA, NVMe-oF, NVLink, HIP, Barex, CXL, and Ascend-family transports
요구 사항, 설치 및 빠른 시작
사용 정보
모델 호환성 및 사용 사례
Mooncake is infrastructure rather than a model runtime tied to one model family. The supplied data mentions use with Kimi, Kimi-K2, DeepSeek deployments, GLM 5.2 FP8, LLaMA3-70B benchmark sizing, large mixture-of-experts systems, and multimodal pipelines. This does not establish universal compatibility with every model. Exact supported model lists and version constraints are not stated in the repository metadata.
라이선스 및 위험 참고 사항
The repository metadata identifies the license as Apache-2.0 and provides https://api.github.com/licenses/apache-2.0. Users should review the repository's license file and the licenses of integrated runtimes, drivers, transports, and serving frameworks before deployment.
릴리스 및 유지 관리
The supplied update history reports that Transfer Engine was open-sourced on November 28, 2024 and Mooncake Store on March 7, 2025. It also records integrations or adoption involving vLLM, SGLang, TensorRT-LLM, LMCache, LMDeploy, NIXL, vLLM-Ascend, TorchSpec, and other projects. The repository metadata reports 6,132 stars, 1,047 forks, 462 open issues, a creation timestamp of 2024-06-25T05:21:05Z, and that the repository is not archived. No formal latest release version is stated.