PAI-PPU-V1.4.0 release notes

1. Overview

To help you quickly use ml.gp7vf.16.40xlarge (Zhenwu 810E) resources on PAI Serverless, PPU provides an official PAI Serverless image. This image combines components from PPU, high-speed networking, and PAI to provide an out-of-the-box experience and optimal performance.

2. Notes

• This image is available to invited customers only. You must comply with confidentiality obligations.

• The PAI-PPU training image runs only on the PAI Serverless platform (including DSW, DLC, and EAS).

• The PAI-PPU inference image runs only on PAI Serverless - EAS.

3. Major updates

• Updated the PPU SDK to version 1.4.0-hotfix2.

• Updated ACCL-P to version 1.4.0-6.

• Integrated Open-MMLab components into the training image for autonomous driving scenarios.

• Added a CUDA 11 Legacy training image for autonomous driving scenarios.

• Integrated PAI-Bladellm-0.9.0RC13 into the inference image, significantly improving performance compared to open-source inference frameworks.

• Fixed two known issues in PPU SDK 1.4.0:

  • Fixed an issue where TransformerEngine 1.11 could cause a CPU memory leak in cuBLAS under specific conditions.

  • Fixed an issue where model training performance degraded when PP>1 in Megatron-Core.

4. Image contents

4.1 Training images

4.1.1 General-purpose training

4.1.2 Autonomous driving training

4.1.3 Legacy scenarios

4.2 Inference images

5. Obtaining images

5.1 From the PAI console

• In your PAI workspace, go to the **Images** page and select the **PAI Official Images** tab to view the official PAI-PPU images.

• When you create an instance in a PAI service, such as a DSW instance, select the **Official Images** tab. You can then filter by chip type to find the official PAI-PPU images.

5.2 Image addresses

6. Known issues

No known issues.

7. Appendix 1: PPU SDK core updates

7.1 Compatibility

7.2 Core components

7.2.1 Core driver updates

• Added support for a circular print buffer to enhance the printf function in CUDA device code.

• Added support for the stream memory operation v2 API.

• Added support for edge data APIs in graph management.

• Added support for batch memory operation node APIs in graph management.

7.2.2 Core acceleration library updates

• CUTLASS 3: Added support for the group GEMM persistent strategy.

• Improved FlashAttention performance on Zhenwu 810E by up to 85%.

• acext: Added support for a16w8 sub-channel quantization.

• MoE: Added support and optimizations for a16w4 quantization.

• xFormers: Fixed an INF issue in attention mask scenarios.

• RTC: Added Run-Time Compilation (RTC) support for group convolution and GEMV.

• Added more precompiled instances to reduce RTC frequency.

• BLAS: Added support for rank-1 and rank-2 Level 2 APIs.

• BLAS: Fixed incorrect GEMM behavior when k=0.

• BLAS INT8 GEMM: Added support for INT8 output and for NN, TN, and TT layouts.

• BLAS: Added support for cublasSgetriBatched.

• conv: Improved performance by over 100x for workloads with wgrad inputs larger than 4 GB.

• conv: Fixed a 1D SpatialTF exception issue.

• acFFT: Added support for C2C, D2Z, and Z2Z transformations and several helper APIs.

• Solver: Added support for Cholesky decomposition and solving, QR decomposition, and the Jacobi method for SVD decomposition.

7.2.3 Core compiler updates

• Added support for the system-level reserved shared memory feature.

• Added support for Triton versions 2.3.x and 3.0.x.

• Supports GCC host compiler versions 5.5 to 12.3.

• ppu-gdb:

  • Optimized the display length of kernel-managed names in layout asm mode.

  • Added support for conditional breakpoints on internal variables, such as blockIdx, and on registers.

  • Added support for GDB Python extensions for Python versions 3.6 to 3.10.

  • Added support for single-hit and multi-hit triggering for device kernel breakpoints. The default is single-hit.

7.2.4 Core PPU-SMI updates

• Added support for querying XID error code descriptions.

• Added support for querying the status of performance counters.

• Added the gpm subcommand for querying and setting GPM status.

• Added support for querying the tidal mode status.

• Added support for querying the product architecture and minor number.

• Added support for setting and querying the Multi-Process Service (MPS) mode.

7.2.5 Core DCGM updates

• Added support for field IDs related to the send/receive rate of each Inter-Chip Network (ICN) link.

• Updated the list of supported field IDs.

7.2.6 Core Asight System updates

• Improved the UI style and layout, and introduced additional tab styles for navigation.

• asys now supports specifying the call stack depth for CPU and Python profiling.

• asys now supports collecting trace data from specific application processes.

• asys now supports collecting basic PPU information, such as frequency and temperature.

• asys now supports shorter sampling intervals for CPU call stack collection.

• asys stats now supports exporting statistics and traces for HGGC kernel grids and blocks.

• asys now supports collecting Python backtraces for HGGC.

• asys now supports collecting Python backtraces for memory operations.

• asys can now trace Python functions across all threads in a Python process.

• The Timeline View now supports custom colors for HGTX ranges.

• The Timeline View now updates the PPU node time percentage when you apply a filter.

• The Timeline View now displays metrics for RDMA network interface cards.

• The Timeline View now displays dependencies between PPU activities.

• The Timeline View can now display HGGC Graph information in a standalone view.

• The Timeline View now supports the "Mark Timeline" feature.

• The Timeline View now uses different colors for different types of video timelines.

• HGGC Launch API calls in the Timeline View now display the kernel name.

• You can now close a report tab while the report is loading.

• The Events View now includes a Tile Mode and provides enhanced autocomplete for search history.

• The Function View now includes flame and icicle graphs.

• HGTX range summaries now support specifying lists of processes and threads.

8. Appendix 2: ACCL-P core updates

The unique interconnect of the Zhenwu 810E chip requires specific environment variables for different parallelism strategies. The chip index changed in ACCL-P 1.4.0-6, so you must adjust these environment variables.

# Basic environment variables
export NCCL_DEBUG=INFO
export NCCL_IB_DISABLE=1
export NCCL_SOCKET_IFNAME==eth0
export NCCL_IB_HCA=

Appendix 3: FAQ

Check the PPU SDK version

The PAI-PPU-V1.4 image integrates PPU SDK 1.4.0-hotfix2. PPU SDK 1.4.0 and PPU SDK 1.4.0-hotfix1 have known issues and are not recommended. You can check the PPU SDK version as follows:

After the image starts, run the ppu-smi -q command and check the "SDK Version" field in the output:

root@122d8d7a7e37:~# ppu-smi -q

==============PPUSMI LOG==============

Timestamp                                   : Tue Dec  31 19:13:36 2024
Driver Version                              : 1.3.2
HGGC Version                                : 12.3
SDK Version                                 : 1.4.0-6974ba

Attached PPUs                               : 16
PPU 00000000:01:00.0
    Product Name                            : alixpu
    Persistence Mode                        : Disabled
    MIG Mode
        Current                             : N/A
        Pending                             : N/A
    Serial Number                           : N/A
    PPU UUID                                : PPU-7f53d39f-ce6e-dc78-c3d4-4c18653c19c0
    PCI
        Bus                                 : 0x01
        Device                              : 0x00
        Domain                              : 0x0000
        Device Id                           : 0x1E0410DE
...

The "SDK Version" output corresponds to the actual PPU SDK version as follows: