PAI-PPU-V1.5.0 official image release notes
1. Image overview
To help you quickly enable ml.gp7vf.16.40xlarge resources on PAI Serverless, PPU has released an official image. This image integrates capabilities from PPU, the high-speed network, and PAI to deliver optimal performance out of the box.
2. Notes
The PAI-PPU training image is supported only on the PAI Serverless platform, which includes the DSW, DLC, and EAS modules.
The PAI-PPU inference image is supported only in PAI Serverless - EAS.
When you use PPU series resources in PAI, you can configure the driver settings. For the PAI-PPU-V1.5.0 image, we recommend using driver version 1.5.0.
3. Key updates
Upgraded the PPU SDK to version 1.5.0.
Upgraded the EIC SDK to version 1.3.7.1-v2504.3 for compatibility with PPU SDK v1.5.0.
Updated the image stack to CUDA 12.6 + PyTorch 2.6.0.
Updated the DeepSeek-R1 inference solution to SGLang 0.4.4.post3 and vLLM 0.7.3. vLLM 0.7.3 adds support for FlashMLA and optimizes the inference performance for A8W8 quantized operators. We recommend using DeepSeek-R1-int8 + vLLM 0.7.3 + FlashMLA as a best practice. The following are the performance test results:
(Tested under laboratory conditions. The results are for reference only and should not guide your instance selection.)
4. Image contents
4.1 Training images
pai-pg1-training-1.5.0-ubuntu | pai-pg1-training-1.5.0-ubuntu-latest | |
Scenarios | CUDA 12.3 + PyTorch 2.4 | CUDA 12.6 + PyTorch 2.6.0 |
Operating system | Ubuntu 22.04 | Ubuntu 24.04 |
PPU SDK version | 1.5.0 | 1.5.0 |
EIC SDK version | 1.3.7.1-v2504.3 | 1.3.7.1-v2504.3 |
Compatible CUDA version | 12.3 | 12.6 |
Python version | 3.10 | 3.10 |
PyTorch version | 2.4.0 | 2.6.0 |
OpenMMLab component versions |
|
|
How to access |
This version includes common OpenMMLab components. You can install other OpenMMLab component versions from the PPU-PIP repository (PPU-V1.5.0 PIP package list).
4.2 Inference images
pai-pg1-inference-1.5.0-vllm0.7.3 | pai-pg1-inference-1.5.0-sglang0.4.4.post3 | |
Scenarios | vllm 0.7.3, with support for FlashMLA | sglang 0.4.4, with support for Reasoning Parser |
Operating system | Ubuntu 24.04 | Ubuntu 24.04 |
PPU SDK version | 1.5.0 | 1.5.0 |
EIC SDK version | 1.3.7.1-v2504.3 | 1.3.7.1-v2504.3 |
Compatible CUDA version | 12.6 | 12.6 |
Python version | 3.10 | 3.10 |
PyTorch version | 2.5.1 | 2.5.1 |
Inference engine version | vllm 0.7.3 | sglang 0.4.4.post3 |
Validated models | DeepSeek-R1/V3 | DeepSeek-R1/V3 |
How to access |
The preceding large model inference images are supported only in PAI-EAS. For DSW and DLC, use the "pai-pg1-training-1.5.0-ubuntu-latest" image and install vllm or sglang by using pip:
pip install vllm==0.7.3+ppu1.5.0.oe
pip install sglang==0.4.4.post3+ppu1.5.0.oe5. Getting the images
5.1 Accessing images directly in PAI
In your PAI workspace, navigate to Images > PAI Official Images on the left to find the PAI-PPU official images.
When you create an instance in a PAI product, such as a DSW instance, select Official Image. Then, under Chip Type, select PPU to filter for the PAI-PPU official images.
5.2 Using inference images from the Model Gallery
In the Model Gallery, select a model from the DeepSeek-R1/V3 series. On the deployment page, select the single-node - GP7V instance type deployment mode to use the PPU series inference images.