概要
The project is designed for enterprise-grade inference, with particular emphasis on Chinese AI accelerators. The repository states that xLLM has been used across JD.com's core retail business and is hosted by the OpenAtom Foundation. Its scope is model inference rather than model training. The supplied data does not describe Agent, MCP or RAG functionality.
主な機能
- Inference support spanning LLM, VLM, DiT and REC model categories
- Optimizations for multiple Chinese AI accelerator families
- Service and computation engine separation
- Online-service and offline-inference workflows
- Hybrid KV-cache management with offloading and prefetching
- C++ implementation
- High-throughput and low-latency inference focus
- Docker images linked from the project README
要件、インストール、クイックスタート
使用方法
モデルの互換性とユースケース
The repository describes support for LLM, VLM, DiT and REC models. Announced model support includes MiniMax-M3, DeepSeek-V4, GLM-5, GLM-4.7, GLM-4.6V, GLM-4.5/GLM-4.6 and VLM-R1. The authoritative compatibility list is linked at https://docs.xllm-ai.com/en/supported_models/. Compatibility details such as supported quantization formats, context lengths and per-hardware model coverage are not stated in the supplied data.
ライセンスとリスクに関する注意事項
Apache License 2.0, identified by SPDX as Apache-2.0. License reference: https://api.github.com/licenses/apache-2.0. Users should review the license text and relevant dependency licenses before redistribution.
リリースとメンテナンス
The README reports donation to the OpenAtom Foundation on 2026-07-06, day-zero MiniMax-M3 support on 2026-06-13 and day-zero DeepSeek-V4 support on 2026-04-24. Earlier announcements include GLM-5, GLM-4.7, GLM-4.6V, GLM-4.5/4.6, VLM-R1 and hybrid KV-cache management. Formal tagged-release details are not stated in the supplied record.