SDK release notes

1.3.1 Firmware

• Supports installation via binary packages (rpm or deb).

• Supports dynamic power management. By default, the PPU firmware dynamically adjusts the core frequency (from 200 MHz to the maximum operating frequency) and operating voltage based on real-time workload, temperature, and power consumption data.

• Supports PPU frequency locking. For details, see PPU SMI Device Management Tool.

• Supports secure firmware signing and dual firmware backup to ensure firmware security and reliability.

• Supports firmware upgrades for the power chip.

• Supports BMC out-of-band management, including device status monitoring and out-of-band firmware upgrades.

1.3.2 Kernel driver

• Supports installation via binary packages (rpm or deb) and runfile packages.

• Supports decoupling the kernel driver from the PPU SDK. You can mix and match any kernel driver and PPU SDK version from v1.0 onwards.

• Supports various hardware configurations, including single-node single-card, single-node 8-card/16-card (ICN interconnect), multi-node multi-card (ICN interconnect), and multi-node multi-card (GDR). Before using GDR, ensure the alixpu-peermem kernel module, included with the PPU kernel driver, is installed on your system.

• Supports PPU fault reporting and handling.

• Supports the auto-reset feature. When the driver detects a kill overtime or cp invalid cmd error, it automatically resets the PPU device to restore normal operation. This feature is enabled by default. You can use PPU SMI to query its status or disable it.

• Supports MPS (Multi-Process Service). You can use the ppudbg --config_submit_mode 1/0 command to enable or disable this feature.

• Supports dynamic scenarios with mixed small-model and large-model workloads in MPS mode.

• Supports the collection of NVML Generic Performance Metrics (GPM), including SM utilization, SM occupancy, Tensor Core utilization, memory bandwidth utilization, PCIe read/write rates, and ICN link read/write rates.

• Supports MIG (Multi-Instance GPU), which allows a PPU to be partitioned into up to eight instances. Note that the ICN interconnect feature is unavailable when MIG is enabled.

• Supports full-card passthrough virtualization. You can unbind the PPU driver on the host and pass the device directly to a virtual machine. When using this feature, ensure ICN is isolated between different VMs.

• Supports hot upgrades of the host driver in SR-IOV virtualization mode, allowing you to update the host PPU driver while services are running in virtual machines.

• Supports hot migration of virtual machines in SR-IOV virtualization mode. This feature is limited to single-card setups; hot migration with ICN interconnect is not yet supported.

• Adds official support for single-node 16-card Zhenwu 810E configurations and optimizes the default PPU device order (sorted by physical slot from the rear of the chassis).

• Adds GPM collection for L1/L2 cache hit rates, which can be queried using the DCGM tool.

• Adjusts the logical ID numbering rule for 16-card Zhenwu 810E devices to improve interconnect performance.

• Adds support for querying the status of dynamic mixed workloads in MPS mode.

• Adds support for querying Xid error information.

• Adds support for querying and managing the GPM status (enable/disable).

• Disables Xid error reporting for HBM parity errors.

1.3.3 User-mode driver and runtime

• Compatible with most CUDA Runtime APIs (cudaXXX) and CUDA Driver APIs (cuXXX).

• Adds more detailed error logs for Xid 896 errors to aid in diagnosis.

• Optimizes the performance of 2D/3D cudaMemcpy and cudaMemset operations.

• Supports the Stream Memory Operation APIs v2.

• Supports graph management APIs for edge data.

• Supports basic texture features.

• Supports basic texture features.

• Optimized PPU performance when CUDA_DEVICE_MAX_CONNECTIONS=1.

• Supports the cuCtxCreate_v3 API.

• Ensures that printf output from device code is correctly displayed after an exception occurs.

• Supports the HGGC_EXCLUSIVE_STREAMS environment variable for mapping stream groups to different hardware queues.

• Fixes an issue where CUDA Graph does not support cooperative kernel nodes.

1.3.4 Compiler

• This release provides a compiler for an extended C/C++ language based on the Clang/LLVM framework. This compiler is designed for the PPU architecture, supports a host/device mixed programming style, and is fully compatible with the CUDA C/C++ programming language specification.

• Provides a rich set of compilation modules that allow developers to flexibly construct compilation workflows and integrate JIT compilation capabilities through API calls.

• Provides a comprehensive suite of development and debugging tools, including hgobjdump, PPU MemCheck, PPU GDB, sanitizer libraries, and hgprune, to improve developer productivity.

• Supports the system-level reserved shared memory feature.

• Supports host compiler versions from GCC 5.5 to 14.2 and from Clang 9 to 18.

• Supports Triton versions 2.3.x, 3.0.x, 3.1.x, and 3.2.x.

• Provides partial compatibility with sparse MMA PTX instructions.

• Supports some Texture-related APIs and inline PTX instruction features.

1.3.5 Acceleration libraries

Compute acceleration libraries

• Proprietary compute libraries include: acdnn, acblas, acfft, acsolver, and acrand.

  • acdnn supports the following operators:

    Conv	BatchNorm	Pooling	Softmax
    Activation	CTCLoss	Dropout	LRN
    LSTM	GRU	MultiHeadAttn	Tensor Ops
    SpartialTransform	Backend fusion

  • acblas supports the following operators:

    Level-1 series operators	Gemv	Gemm	Matmul + epilogue
    MatrixTransform	trsm	getrfBatched	getrsBatched
    geqrfBatched	gelsBatched

  • acfft supports R2C/C2R/C2C/D2Z/Z2Z + FFT/iFFT transforms.

  • acsolver supports matrix LU decomposition/solve, Cholesky decomposition/solve, QR decomposition, SVD decomposition, and eigenvalue decomposition.

  • acrand supports:

    • Pseudo-random number generators: XORWOW, MRG32K3A, PHILOX4_32_10.

    • Data distributions: Default, Uniform, Normal, LogNormal.

  • The Acext quantization library supports:

    • Various kernel variants for A16W8/A16W4 with PerChannel/GroupWise quantization.

    • Various kernel variants for A8W8 with PerChannel/PerToken quantization.

    • Acceleration kernels for WeightonlyBatchedGemv with small batch sizes.

    • The following MoE types: FP16/BF16, and a8w8 PerChannel/PerToken.

• Key updates:

  • Acext: Adds tuning support for DeepSeek MoE.

  • Adds an offline RTC cache mechanism to mitigate performance issues caused by RTC compilation.

  • blasLt: Adds support for Atomics Synchronization in matmul.

  • blas: Further performance tuning for large-model inference scenarios, with operator performance improvements of up to 20%.

  • conv: Adds support for new combinations of activation and weight formats.

  • conv: Adds heuristic-based selection logic for Tensor Core and Vector Core algorithms.

  • conv: Optimizes group conv performance, improving operator performance by up to 33x.

  • conv: Resolves slow performance in non-TensorFlow scenarios when a tensor exceeds 4 GB, improving operator performance by up to 230x.

  • solver: Adds support for new eigenvalue decomposition interfaces: syevd, sytrd, ormtr, and orgtr.

Interconnect acceleration library

• Compatible with most NCCL APIs (ncclXXX) and environment variables.

• Supports typical collective operators such as AllReduce, AllGather, ReduceScatter, Broadcast, Reduce, Send, and Recv.

• Supports most NCCL APIs (ncclXXX) and environment variables.

• Supports communication between multiple nodes via RDMA (GDR & non-GDR), sockets, and ICN.

• Performance optimizations:

  • Optimizes alltoall performance in multi-node Zhenwu 810E environments, improving peak throughput by 20% to 60% compared to previous versions. Achieving this performance gain requires configuring specific environment variables.

  • Improves operator performance for input or output buffers that are not 64-byte aligned, improving allgather performance by approximately 40% in such scenarios.

  • Resolves some inconsistencies between the performance of communication operators in models and their standalone benchmark performance. Key operators like allreduce show an approximately 20% improvement in some scenarios.

  • Enhances the topology awareness of oneShot and twoShot kernels for different direct-connect topologies and optimizes performance parameters accordingly.

• Enhanced debugging features for multi-node, multi-card setups:

  • Enhances monitoring for PPU device kernel execution state.

    • Eliminates the monitoring solution's overhead on kernel execution time.

    • Adds automatic cleanup of old kernel execution state dump files.

    • Adds a configurable PCCL dump directory.

    • Optimizes error message display from monitoring threads to reduce interference with the main task.

  • Adds support for dumping the state of single-node oneShot and twoShot kernels.

  • Adds checks for ICN speed and topology correctness on multi-card Zhenwu 810E products.

• Adds multi-node support for environments with EIC NICs, for use with the EIC SDK.

• Adds support for the splitShare feature in communicator-splitting scenarios.

• Adds initial support for using the DMA engine for multi-card transfers in P2P transport.

• Adds support for ncclNet plugin v5 and v6.

• Adds support for the new log subsystem level NCCL_CALL, which prints the PCCL APIs called by the program and their parameters.

• Enhances hgtx information for PCCL kernels to include the communicator's transport type and kernel input/output buffer information.

• Fixes some issues with the multi-node, multi-card ICN scale-out feature in multi-node non-ICN connected environments.

• Fixes an issue where alltoallV would hang in some use cases.

• Fixes an issue that caused incorrect results when NCCL_MAX_P2P_NCHANNELS was set to a value smaller than NCCL_MAX_NCHANNELS.

• Fixes a potential internal buffer overflow issue in debug log printing.

• PCCL tools:

  • DeviceOrderSearch tool:

    • Adds the DeviceOrderSearch tool to determine the optimal visible device order for Megatron framework model training tasks on Zhenwu 810E machines, based on the parallel training configuration.

  • pccl-perf:

    • Supports AllReduce, AllGather, ReduceScatter, AlltoAll, Broadcast, and Reduce.

    • Supports a new method for timing PCCL device kernels based on the hgpti callback mechanism. For more information, see the documentation.

    • Removes the dependency on libmpicxx.so.

  • p2pBandwidthAndLatencyPerf tool enhancement:

    • Supports evaluation of ICN P2P interconnect bandwidth and latency for Zhenwu 810E products on various multi-card interconnect topology servers.

  • pccl-check-tools:

    • Supports readiness checks for Zhenwu 810E products in multi-card or multi-node, multi-card scenarios.

1.3.6 Video and image codec acceleration

Compatible with the NVIDIA Video Codec SDK, including cuvid (decode), nvenc (encode), and nvjpeg. Also compatible with the NVIDIA 2D Image NPP interface. This allows out-of-the-box hardware acceleration for high-level frameworks like FFmpeg, OpenCV, DALI, and PyAV (used by torchvideo, torchvision, etc.), and TorchCodec without requiring code modifications.

Video decode

• Supports NVIDIA cuvid decoder.

• Codecs:

  • HEVC (H.265) - ITU-T Rec. H.265 (04/2013), ISO/IEC 23008-2

    • Main Profile, Level 5.1, High Tier

    • Main10 Profile, Level 5.1, High Tier

    • Main Still Profile

  • VP9 - vp9-bitstream-specification-v0.6-20160331-draft

    • Profile 0, 8-bit

    • Profile 2, 10-bit

  • AVC (H.264) - ITU-T Rec. H.264 (03/2010) / ISO/IEC 14496-10

    • Main Profile, levels 1 - 5.2

    • High Profile, levels 1 - 5.2

    • High 10 Profile, levels 1 - 5.2

    • Baseline Profile, levels 1 - 5.2

  • AV1 Bitstream & Decoding Process Specification Version 1.0.0 with Errata 1

    • Main Profile, Level 5.1

  • AVS2

• Resolution up to 8192x8192

• Up to 192 FHD streams

Video encode

• Supports nvenc.

• Codecs:

  • AVC(H.264)：Spec Version 12:ISO/IEC 14496-10 / ITU-T Rec. H.264 (03/2010)

    • Baseline Profile, levels 1 – 5.2

    • Main Profile, levels 1 - 5.2

    • High Profile, levels 1 – 5.2

    • High 10 Profile, levels 1 - 5.2

  • HEVC(H265)：ITU-T Rec. H.265 (04/2013), ISO/IEC 23008-2

    • Main Profile, Level 5.1, High Tier

    • Main10 profile, Level 5.1, High Tier

    • Main Still Profile

  • AV1 Bitstream Specification Version 1.0.0 with Errata 1

    • Main Profile, Level 5.1

• Supports resolutions up to 4K.

• Supports input in RGB format (converted to YUV420 via inlinePP).

• Supports crop, scale, and rotate operations with inlinePP.

• Up to 32 FHD streams

JPEG

• Supports nvjpeg decoder and encoder.

• Resolution up to 32Kx32K

• Supports RGB format input and output with inlinePP.

• Supports crop, scale, and rotate operations with inlinePP.

• Up to 960 FPS at UHD resolution.

Image processing

• Supports NVIDIA 2D Image NPP.

1.3.7 Software tools

• To meet the monitoring needs of large-scale cloud computing clusters, this release provides the following PPU management and monitoring tools and libraries for integration into customer cluster operations and monitoring systems:

  • The PPU device management tool ppu-smi is similar to nvidia-smi. For details, see PPU SMI Device Management Tool.

  • New features in ppu-smi v1.5 include:

    • Addresses known issues.

    • Adds a description for querying the version information of SDK components.

  • The data center management and monitoring tool PPU DCGM is similar to NVIDIA DCGM. For details, see DCGM Management and Monitoring Tool.

  • New features in PPU DCGM v1.5 include:

    • Adds instructions for using PPU DCGM in a Docker environment.

  • hgml, similar to NVIDIA NVML.

• We provide the PPU performance analysis tools Asight Systems and Asight Compute (similar to NVIDIA Nsight Systems and Nsight Compute). You can use them to analyze performance for single-node and multi-node training and inference scenarios.

  • Asight Systems is a low-overhead, system-level performance analysis tool. It collects system events, CPU and PPU activity, API execution times and related call stacks, NVTX, and CPU/PPU activity correlations, then presents them in a unified Timeline View. By using the Timeline View, developers can analyze CPU/PPU workloads and their relationships, identify performance bottlenecks, and ensure coordinated operation and maximum parallelism between the CPU and PPU. For details, see Asight Systems Application Performance Analysis Suite.

  • New features in Asight Systems v1.5 include:

    • The asys stats command supports HGTX PPU projection summary statistics.

    • The asys stats command supports statistics for HGGC PPU memory tracking data volume and time consumption.

    • PPU kernel-related statistics in the asys stats subcommand support adding an HGTX range prefix.

    • The HGTX Range Summary in the asys stats subcommand supports filtering by process.

    • PPU trace-related statistics in the asys stats subcommand support specifying a list of PPU devices for which to collect statistics.

    • The asys stats subcommand supports 2 viewing modes for Device Memory Usage Details and 4 viewing modes for Device Memory Usage Summary.

    • The PPU time utilization analysis in the asys analyze subcommand supports a selection mode for specifying a time range.

    • Simplifies the description text in the Asight Systems expert system and statistics system, and adds links to the documentation.

    • Asight Systems now supports limiting the size of generated report files.

    • Asight Systems supports enabling the collection of device and pinned memory usage via the --hggc-memory-usage option.

    • Supports collecting CPU-side dynamically allocated memory usage, and supports memory leak analysis, memory usage statistics, and allocation count statistics.

    • Supports collecting compute resource information associated with the HGGC context when MPS mode is enabled.

    • Memory View supports four viewing modes and flame graphs.

    • Timeline View optimizes the display of PPU activity dependencies.

    • Timeline View supports a grouping feature, allowing you to add timeline rows of interest to a Group View for separate display.

    • Timeline View supports Device Memory Timeline to help users analyze memory usage by module. It supports three modes: normal, group, and group coloring.

    • Timeline View supports a hierarchical display of communication operators.

    • Timeline View displays different kernels in different colors.

    • The Timeline View shows the CPU-side heap memory timeline.

    • The time range tooltip displays the percentage of active and idle time for a PPU.

    • In the Timeline view, hold the Shift key while selecting a time range to snap the selection to the nearest start or end point.

    • Timeline View lets you select multiple timeline rows with Shift or Ctrl.

    • Optimized the performance of Timeline View to fix GUI lag when opening very large reports.

    • The HGGC API supports a mixed display of C and Python call stacks.

• Asight Compute is a kernel performance analysis tool. It collects PPU hardware performance counters and combines them into a series of performance indicators called metrics. The graphical user interface (GUI) displays these metrics from various perspectives to help you analyze and optimize the kernel in depth. For more information, see Asight Compute Kernel Analyzer.

• New features in Asight Compute v1.5 include:

  • The Memory Chart on the Details page now supports display by transfer size or throughput.

  • Tooltips for connection links in the Memory Chart on the Details page now show link utilization.

  • Tooltips on the Details page can now be pinned. Once pinned, you can select and copy text from the tooltip.

  • Added Stack Size Per Thread to Launch Statistics on the Details page.

  • Added Driver SharedMemory Per Block to Launch Statistics on the Details page.

  • Updated the Baseline button layout to clarify the function of each button.

  • On the Source page, you can now click branch instructions to navigate to them. The page also supports highlighting all search results.

  • Improved source code resolution on the Source page. The page now automatically resolves other files within the same directory structure.

• Released the PPU debugging tool PPU-GDB, which lets you debug both GPU and CPU code in the same application. The latest updates are:

  • Optimized the display length of kernel managed names in layout asm mode.

  • Added support for conditional breakpoints on internal variables, such as blockIdx, and on registers.

  • Added support for the GDB Python extension feature (for Python versions 3.6 to 3.10).

  • Added support for trigger behaviors for device kernel breakpoints: single trigger and multiple triggers. The default is single trigger.

• hgprune is a PPU prune tool used to strip unnecessary information from binary files.

• Released the hgobjdump PPU Binary tool to extract device information from a binary.

• Released hgprune, a PPU Prune tool that extracts device-related information from a binary.

1.5.1 KMD compatibility

Version 1.5 is forward compatible with kmd/mcu_fw from versions 1.3 and 1.4.

1.5.2 Firmware compatibility

Zhenwu 810E requires mcu_fw version 1.2.1 or later.

1.5.3 SDK compatibility

SDK v1.5 introduces four types of adjustments to files, programming APIs, and the compilation environment. These changes can cause incompatibilities when components from previous SDK versions, or components compiled with them, coexist with v1.5. The adjustments are as follows:

• The libgomp.so and libomp.so files are removed. The SDK now depends on the libgomp.so and libomp.so library files provided by the operating system (OS) environment and the host compiler, such as gcc or clang.

This change can cause older programs to fail in the v1.5 SDK environment, reporting that the required .so files are missing.

• The compilation command option format is changed from -alippu-xxx to -ppu-xxx.

These options are mainly used for custom compilations. This change can cause compilation errors when compiling mixed-version source code or during Just-In-Time (JIT) execution.

• Several extended PTX instructions are adjusted:

  1. The alippu.mma.xxx PTX instruction is changed to ppu.mma.xxx.

  2. The alippu.ldmatrix.xxx PTX instruction is changed to ppu.ldmatrix.xxx.

  3. The cp.async.aiu.xxx PTX instruction is changed to ppu.cp.async.aiu.xxx.

• In the SDK header files, the custom type __ali_bfloat16 is changed to __ppu_bfloat16.

The latter three changes cause compilation issues with some open-source PPU code libraries, such as vllm, cutlass, flash-attn, and flashinfer. Pull the code libraries that are adapted for the v1.5 changes.

In summary, these four adjustments cause compilation and runtime compatibility issues when source code is not updated or when runtime components are partially upgraded. This behavior was confirmed during internal SDK compatibility testing. Deploy your compilation and runtime environments using the complete v1.5 SDK.

Test conclusions

For this test, six commonly used user .whl files were randomly selected across six scenarios, including the Asys and Acu tools. All tests failed as expected.

• .whl files compiled with older SDK versions are highly likely to be incompatible with the SDK 1.5 image. As a workaround, manually install the libomp library to continue. However, the unit test (UT) pass rate is not guaranteed.

• .whl files compiled with SDK 1.5 are highly likely to be incompatible with the SDK 1.4 image, and the UTs are highly likely to fail.

• JIT-type libraries, such as Triton, are confirmed to be incompatible when used with cross-version SDKs.

• Compiling and installing .whl files with mixed SDK versions is highly likely to cause incompatibility. For example, do not use libraries compiled with mixed SDKs in an SDK 1.4 image. Likewise, do not compile libraries from older versions in an SDK 1.5 image.

• The SDK 1.5 image is compatible with Asys and Acu reports from SDK 1.4.

1.5.4 asight file compatibility

The output files of asys and acu in v1.5 are compatible with the asys and acu client tools from v1.4.

2.3.1 Supported open-source frameworks and libraries

Notes:

• PyTorch open-source support policy:

  • PyTorch versions 2.1 and earlier are aligned with NGC releases.

  • PyTorch versions 2.1.2 and later are aligned with the PyTorch open-source community release versions.

• TensorFlow open-source support policy:

  • The Google open-source community version of TensorFlow 1.12 supports only CUDA 9, and version 1.15 supports only CUDA 10. For more information, see https://www.tensorflow.org/install/source?hl=en#gpu_support_3.

    The released TensorFlow 1.15 is based on the open-source version from NVIDIA NGC nv22.04. For more information, see https://docs.nvidia.com/deeplearning/frameworks/support-matrix/index.html#unique_1997996242.

  • TensorFlow 1.12 is based on PAI-TF 1.12.

  • TensorFlow 2.4 is based on NVIDIA NGC nv21.02.

  • TensorFlow versions 2.6 and 2.7 are aligned with the Google open-source community release versions.

  • TensorFlow 2.8 is based on NVIDIA NGC nv22.04.

  • TensorFlow versions 2.10 and later are aligned with the Google open-source community release versions.

• Frameworks and libraries marked with an asterisk (*), such as torchmetrics(*), are fully aligned with their community open-source versions. They are supported for functionality only and have not been optimized for performance.

• Other open-source frameworks and libraries are mostly aligned with their community release versions, with a few exceptions that are aligned with NGC. For more information, see the usage guide for each framework or library.

• The PPU SDK is compatible with the compilation, basic functionality, and unit tests of the listed open-source frameworks and libraries. Due to resource constraints, some issues found during testing remain unresolved in the v1.5 release. For unresolved issues, see the known issues section.

Open-source framework compatibility