PAI-PPU-V1.5.2 official image release notes
Image overview
The PAI-PPU official image enables the ml.gp7vf.16.40xlarge (Zhenwu 810E) resource on PAI Serverless. It combines the PPU, high-speed networking, and PAI layers to deliver optimal performance out of the box.
Important notes
The PAI-PPU training image can only be used on the PAI platform (DSW, DLC, and EAS).
The PAI-PPU inference image is supported only in PAI-EAS.
This official image update does not include a driver update. We recommend using driver version 1.5.0 with the PAI-PPU-V1.5.2 image.
This official image does not currently support DeepEP and sailSHMEM. A support timeline has not yet been determined.
Important: With this update, the training image now uses the aiext-pypi source. Ensure your DSW or DLC instance is launched with a RAM role and that your primary account UID is on the allowlist. To be added to the allowlist, contact your account manager or a PAI PDSA.
What's new
Available regions: China (Ulanqab), China (Beijing), China (Shanghai), and China (Hangzhou).
Upgraded the PPU SDK to version 1.5.2. This version does not currently support DeepEP and sailSHMEM.
The EIC SDK has not been updated in this release.
Important: The PIP source for the training image is now aiext-pypi.mirrors.aliyuncs.com. This new source offers the following advantages over the previous art-pub.eng.t-head.cn source:
Authentication no longer requires a username and password. Access now requires your primary account to be on an allowlist, following the same process as for the PPU documentation library. (To be added to the allowlist, contact your account manager or a PAI PDSA).
Natively supports access from a VPC private network (100.x.x.x address range), eliminating the need for PrivateLink configuration.
The inference image continues to use the t-head source.
The acext operator library in the inference image has been updated again, delivering significant performance improvements for the DeepSeek-R1-int8 and Qwen3-235B-A22B-int8 models. We will provide specific performance data in a future update.
Other key updates:
Fixed an issue where Megatron-LM training would hang when saving weights in the torch_dist format.
Fixed a bug that caused the vLLM inference engine to generate garbled responses when prefix caching was enabled.
Fixed an issue where the error log "Failed to create /dev/alixpu node" appeared when compiling vLLM in a non-PPU environment.
Image contents
Training images
pai-pg1-training-1.5.2-ubuntu | pai-pg1-training-1.5.2-ubuntu-latest | |
Use cases | CUDA 12.3 + PyTorch 2.4 | CUDA 12.6 + PyTorch 2.6.0 |
Operating system | Ubuntu 22.04 | Ubuntu 24.04 |
PPU SDK version | 1.5.2 | 1.5.2 |
EIC SDK version | 1.3.7.1-v2504.3 | 1.3.7.1-v2504.3 |
Compatible CUDA version | 12.3 | 12.6 |
Python version | 3.10 | 3.12 |
PyTorch version | 2.4.0 | 2.6.0 |
Open-MMLab component versions |
|
|
Access |
Inference images
vllm:0.8.5-xpu1.5.2 | sglang:0.4.6.post1-xpu1.5.2 | |
Use cases | vLLM 0.8.5, integrated with the acext operator library | SGLang 0.4.6.post1, integrated with the acext operator library |
Operating system | Ubuntu 24.04 | Ubuntu 24.04 |
PPU SDK version | 1.5.2 | 1.5.2 |
EIC SDK version | 1.3.7.1-v2504.3 | 1.3.7.1-v2504.3 |
Compatible CUDA version | 12.6 | 12.6 |
Python version | 3.12 | 3.12 |
PyTorch version | 2.6.0 | 2.6.0 |
Inference engine version | vLLM 0.8.5 | SGLang 0.4.6.post1 |
Verified models | DeepSeek-R1/V3, Qwen3 | DeepSeek-R1/V3, Qwen3 |
Access |
The preceding large model inference images are supported only in PAI-EAS and not in DSW or DLC. We recommend the following approach:
Use the pai-pg1-training-1.5.2-ubuntu-latest image, which is supported in DSW, DLC, and EAS. Then, install the latest version of vLLM or SGLang by using pip:
pip install vllm pip install sglang
Obtaining images
Obtain images directly in PAI
In your PAI workspace, navigate to the Images tab to find a list of PAI official images, including the PAI-PPU images.
When you create an instance in any PAI sub-product, such as a DSW instance, you can select an official image and filter by Chip Type to quickly find PAI-PPU official images.
Use an inference image from Model Gallery
In Model Gallery, on the deployment page, select the Single Machine - GP7V Instance deployment method to use PAI-PPU series inference images.
Known issues
These images do not yet support the PPU version of DeepEP and sailSHMEM.
Running Qwen2.5-omni with the vLLM inference image triggers the ModuleNotFoundError: No module named 'vllm.vllm_flash_attn.layers' error. This is a known issue in vLLM 0.8.5 that will be fixed in a future update to the PPU version of vLLM.