English | 简体中文
vLLM is a fast and easy-to-use library for LLM inference and serving. Originally developed by the Sky Computing Lab at UC Berkeley, vLLM has evolved into a community-driven project jointly maintained by academia and industry.
The open-source image product vLLM large model service provided by this project comes pre-installed with the vLLM inference framework and its related runtime environment, along with deployment templates. Follow the usage guide to easily enjoy an efficient "out-of-the-box" experience.
System Requirements:
- CPU: 8 vCPUs or higher
- RAM: 16GB or larger
- Disk: At least 40GB
Register a Huawei Account and Activate Huawei Cloud
| Image Specifications | Feature Description | Notes |
|---|---|---|
| vLLM-v0.8.3-Kunpeng | Deployed on Kunpeng Cloud Server + Ubuntu 24.04 64bit / Huawei Cloud EulerOS 2.0 64bit |
- Submit questions via GitHub Issues or contact Huawei Cloud Marketplace support for this product
- Explore other open-source images at open-source-image-repos
- Fork this repository and submit pull requests
- Synchronize updates to README.md based on your open-source image information