अवलोकन
The platform is designed for development teams, IT organizations, and service providers delivering Model-as-a-Service. A GPUStack server manages one or more GPU clusters, schedules workloads, selects and configures inference engines, exposes model APIs, and provides operational visibility through integrated Grafana and Prometheus dashboards. The repository is primarily Python and is not archived.
प्रमुख विशेषताएँ
- Management of GPU clusters across on-premises servers, Kubernetes, and cloud providers
- Pluggable inference engines, including vLLM, SGLang, TensorRT-LLM, and custom engines
- Automated engine selection, GPU scheduling, and parameter optimization
- Pre-tuned low-latency and high-throughput operating modes
- Support for LMCache and HiCache extended KV-cache systems
- Support for EAGLE3, MTP, and N-gram speculative decoding methods
- On-demand, SSH-accessible GPU instances
- Failure recovery, load balancing, monitoring, authentication, and access control
- Grafana and Prometheus operational dashboards
- OpenAI-compatible API endpoints
- Metering of token usage and API request rates
आवश्यकताएँ, इंस्टॉलेशन और त्वरित शुरुआत
sudo docker run -d --name gpustack --restart unless-stopped -p 80:80 --volume gpustack-data:/var/lib/gpustack gpustack/gpustack
Inspect startup logs with `sudo docker logs -f gpustack`. Retrieve the initial administrator password with `sudo docker exec gpustack cat /var/lib/gpustack/initial_admin_password`, then open `http://your_host_ip` and sign in as `admin`. The README also provides a Quay.io mirror option. Complete installation guidance is available at https://docs.gpustack.ai/latest/installation/requirements/.
उपयोग
मॉडल संगतता और उपयोग के मामले
The repository describes APIs for LLM, voice, image, and video models and names Qwen3.5-0.8B as its quick-start example. Repository topics also reference DeepSeek, Llama, and Qwen. Engine support includes vLLM, SGLang, TensorRT-LLM, and custom engines. Supported accelerators listed in the README are NVIDIA GPU, AMD GPU, Ascend NPU, Hygon DCU, MThreads GPU, Iluvatar GPU, MetaX GPU, Cambricon MLU, and T-Head PPU. A complete model compatibility list is not stated in the repository metadata.
लाइसेंस और जोखिम संबंधी टिप्पणियाँ
Licensed under Apache License 2.0. The README identifies copyright for 2024-2026 as belonging to the GPUStack authors and states that the software is provided without warranties or conditions except as required by law. License metadata: https://api.github.com/licenses/apache-2.0.
रिलीज़ और रखरखाव
No release version, changelog, or release notes are included in the supplied repository record. Not stated in the repository metadata.