Alibaba Cloud AI Containers (AC2) provides ready-to-use container images for deploying Qwen series large models. Each image bundles all required dependencies — Python runtime, deep learning frameworks, and supporting libraries — so models run consistently across environments.
The images do not include model weight files. Download the weight files separately, or use the download tool bundled in the image. Images expose a web demo interface for external access and can also serve as base images for custom inference services.
Images
Two image types are available:
-
Runtime image — contains the software environment required to run a model: system components and Python dependencies.
-
Quick deployment (devel) image — extends the runtime image with a startup script and a web demo script for one-step deployment.
| Type | Supported model | CPU/GPU | Image address |
|---|---|---|---|
| Runtime image | Qwen 1.8–72B (including quantized models) | GPU | ac2-registry.cn-hangzhou.cr.aliyuncs.com/ac2/qwen:runtime-pytorch2.2.0.1-cuda12.1.1-alinux3.2304 |
| Qwen 1.8–72B | CPU | ac2-registry.cn-hangzhou.cr.aliyuncs.com/ac2/qwen:runtime-pytorch2.2.0.1-alinux3.2304 |
|
| Quick deployment (devel) image | Qwen-Chat-7B | GPU | ac2-registry.cn-hangzhou.cr.aliyuncs.com/ac2/qwen:7b-pytorch2.2.0.1-cuda12.1.1-alinux3.2304 |
| CPU | ac2-registry.cn-hangzhou.cr.aliyuncs.com/ac2/qwen:7b-pytorch2.2.0.1-alinux3.2304 |
Image content
qwen:runtime-pytorch2.2.0.1-cuda12.1.1-alinux3.2304
| Component | Version |
|---|---|
| gradio | 3.41.0 |
| optimum | 1.19.2 |
| auto-gptq | 0.7.1 |
| flash-attn | 2.5.8 |
| tiktoken | 0.5.2 |
| accelerate | 0.26.1 |
| transformers | 4.36.2 |
| PyTorch | 2.2.0.1 |
| CUDA | 12.1.1 |
| Python | 3.10.13 |
| BaseOS | Alinux 3.2304 |
qwen:runtime-pytorch2.2.0.1-alinux3.2304
| Component | Version |
|---|---|
| gradio | 3.41.0 |
| tiktoken | 0.5.2 |
| accelerate | 0.26.1 |
| transformers | 4.36.2 |
| PyTorch | 2.2.0.1 |
| Python | 3.10.13 |
| BaseOS | Alinux 3.2304 |
qwen:7b-pytorch2.2.0.1-cuda12.1.1-alinux3.2304
Inherits all components from qwen:runtime-pytorch2.2.0.1-cuda12.1.1-alinux3.2304.
qwen:7b-pytorch2.2.0.1-alinux3.2304
Inherits all components from qwen:runtime-pytorch2.2.0.1-alinux3.2304.
Operational requirements
CPU images (qwen:runtime-pytorch2.2.0.1-alinux3.2304, qwen:7b-pytorch2.2.0.1-alinux3.2304): no driver requirements.
GPU images (qwen:runtime-pytorch2.2.0.1-cuda12.1.1-alinux3.2304, qwen:7b-pytorch2.2.0.1-cuda12.1.1-alinux3.2304): include CUDA 12.1.1 and require NVIDIA Driver release 530 or later. Driver releases 470 and 525 are also compatible.
GPU compatibility
GPU images include FlashAttention-2, which requires a compatible GPU architecture. The following table shows the GPU architectures supported by FlashAttention-2 and the corresponding Alibaba Cloud GPU instance models.
| GPU architecture | Alibaba Cloud GPU model | FlashAttention-2 |
|---|---|---|
| Ampere | A10 | Supported |
| Turing | T4 | Not supported |
| Volta | V100 | |
| Pascal | P100 and P4 |
If you run a Qwen model on an unsupported GPU architecture, the following error appears:
FlashAttention only supports Ampere GPUs or newerTo resolve this, remove the FlashAttention-2 component from the running container:
pip uninstall -y flash-attnKey features
-
GPU images come pre-installed with optimum, auto-gptq, and flash-attn to support quantized models.
-
The devel image includes a built-in web demo script. For details, see web_demo.py.
Updates
| Date | Release |
|---|---|
| 2024.07 | Released the quick deployment (devel) image |
| 2024.06 | Released the runtime image |