Übersicht
The project is designed for enterprise-grade inference, with particular emphasis on Chinese AI accelerators. The repository states that xLLM has been used across JD.com's core retail business and is hosted by the OpenAtom Foundation. Its scope is model inference rather than model training. The supplied data does not describe Agent, MCP or RAG functionality.
Wichtige Funktionen
- Inference support spanning LLM, VLM, DiT and REC model categories
- Optimizations for multiple Chinese AI accelerator families
- Service and computation engine separation
- Online-service and offline-inference workflows
- Hybrid KV-cache management with offloading and prefetching
- C++ implementation
- High-throughput and low-latency inference focus
- Docker images linked from the project README
Voraussetzungen, Installation und Schnellstart
Nutzung
Modellkompatibilität und Anwendungsfälle
The repository describes support for LLM, VLM, DiT and REC models. Announced model support includes MiniMax-M3, DeepSeek-V4, GLM-5, GLM-4.7, GLM-4.6V, GLM-4.5/GLM-4.6 and VLM-R1. The authoritative compatibility list is linked at https://docs.xllm-ai.com/en/supported_models/. Compatibility details such as supported quantization formats, context lengths and per-hardware model coverage are not stated in the supplied data.
Lizenz- und Risikohinweise
Apache License 2.0, identified by SPDX as Apache-2.0. License reference: https://api.github.com/licenses/apache-2.0. Users should review the license text and relevant dependency licenses before redistribution.
Veröffentlichung und Wartung
The README reports donation to the OpenAtom Foundation on 2026-07-06, day-zero MiniMax-M3 support on 2026-06-13 and day-zero DeepSeek-V4 support on 2026-04-24. Earlier announcements include GLM-5, GLM-4.7, GLM-4.6V, GLM-4.5/4.6, VLM-R1 and hybrid KV-cache management. Formal tagged-release details are not stated in the supplied record.