DALI (v1.5)-ZHENWU PPU Cloud Service(ppu)-阿里云帮助中心

1. Overview

NVIDIA DALI is an open-source framework for parallel data loading during model training, featuring hardware-accelerated encoding and decoding. The T-Head SAIL SDK supports the DALI framework, significantly accelerating model training performance, especially in CPU-bound scenarios.

Official documentation: https://docs.nvidia.com/deeplearning/dali/user-guide/docs/index.html

Source code: https://github.com/NVIDIA/DALI

2. Install DALI

2.1 Docker

The SAIL SDK Docker image comes with DALI pre-installed. The default version is v1.20:

pip list | grep nvidia-dali

2.2 Install from pip

You can install DALI from the T-Head pip source using a whl package. Starting with SAIL SDK v1.4.1, support for DALI v1.44 is also available. Select the whl package that matches your DALI version (v1.20 or v1.44), operating system (Ubuntu or AliOS), and CUDA version.

For example, to install DALI 1.20 on Ubuntu with CUDA 11.6, run the following command:

pip install nvidia_dali_cuda110==1.20.0+ppu202412182301.ce -i https://art-pub.eng.t-head.cn/artifactory/api/pypi/ptgai-pypi_ppu_ubuntu_cu116_index/simple/
# The ppu2024xxxxxxxx placeholder represents the version number for the SAIL SDK v1.4.1 release.

To install DALI 1.44 on Ubuntu with CUDA 12.4, run the following command:

pip install nvidia_dali_cuda110==1.44.0+ppu202412182301.ce -i https://art-pub.eng.t-head.cn/artifactory/api/pypi/ptgai-pypi_ppu_ubuntu_cu124_index/simple/
# The ppu2024xxxxxxxx placeholder represents the version number for the SAIL SDK v1.4.1 release.

Note

DALI v1.44 depends on the NVIDIA nvImageCodec project, https://github.com/NVIDIA/nvImageCodec, which requires CUDA 11.8 or later. Consequently, DALI v1.44 also requires CUDA 11.8 or later.
DALI's dependency on CUDA is limited to the major version number. This means a DALI whl package for CUDA 12.4 works with other CUDA 12.x versions (like 12.1 or 12.6) without requiring a reinstallation. For more information, see the official NVIDIA documentation: https://docs.nvidia.com/deeplearning/dali/user-guide/docs/support_matrix.html.
Starting with SAIL SDK v1.4.1, DALI supports the video loader feature, which provides hardware-accelerated video decoding. If you plan to use the video loader, we recommend using DALI v1.44, as official support for this feature in v1.20 is incomplete. DALI v1.44 includes the following improvements to the video loader over v1.20:
1. Improves VideoReader robustness and its handling of corrupted streams.
2. Fixed an infinite loop issue in the video decoder seek function.
3. Added support for video streams with different bit depths.

2.3 Install from NVIDIA WHL files

Not recommended. You cannot directly use the DALI installation packages provided by NVIDIA.

2.4 Build from source

Download and build the DALI dependencies:

git clone --depth 1 --branch v1.44.0 --recurse-submodules https://github.com/NVIDIA/DALI_deps.git /tmp/DALI_deps
cd /tmp/DALI_deps
bash build_scripts/build_deps.sh

Download and build DALI:

git clone --branch release_v1.44 --recurse-submodules https://github.com/NVIDIA/DALI.git
cd /workspace/DALI
git apply t-head_dali.patch # A patch is required.
mkdir build && cd build
cmake .. -DBUILD_BENCHMARK=OFF -DWITH_DYNAMIC_NVIMGCODEC=OFF -DBUILD_CVCUDA=OFF # Will fix CVCUDA compile issue soon
make -j

The patch includes the following key changes:

third_party/cutlass/include/cutlass/arch/wmma.h:

-#if !(defined(__clang__) && defined(__CUDA__))
+#if !(defined(__clang__) && defined(__CUDA__)) && !defined(__HGGCCC__)

CMakeLists.txt:

-set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} --compiler-options \"-fvisibility=hidden -Wno-free-nonheap-object\" --Wno-deprecated-gpu-targets -Xfatbin -compress-all")
+set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} --compiler-options \"-fvisibility=hidden -Wno-free-nonheap-object -fno-delayed-template-parsing\" --Wno-deprecated-gpu-targets -Xfatbin -compress-all")

3. Enable DALI

Refer to the official NVIDIA documentation for getting started with DALI:

https://docs.nvidia.com/deeplearning/dali/user-guide/docs/examples/getting_started.html

For examples of integrating DALI with popular deep learning frameworks, see the following code samples:

4. DALI in open-source libraries

4.1 DLE

Some models in NVIDIA Deep Learning Examples (DLE) natively support the DALI data loader.

GitHub - NVIDIA/DeepLearningExamples: State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.

To use this feature, specify dali-gpu as the data backend.

The following example shows how to train a ResNet50 model:

python ./multiproc.py --nproc_per_node 8 ./main.py --arch resnet50 -c fanin --label-smoothing 0.1 --data-backends dali-gpu <path to imagenet>

4.2 PyTorch image models

PyTorch image models (timm): https://github.com/huggingface/pytorch-image-models

The following example shows how to train a MobileNetV3 model:

bash ./distributed_train.sh 2 /root/datasets/vision/imagenet --model mobilenetv3_large_100 -b 512 --sched step --epochs 1  --decay-epochs 2.4 --decay-rate .973 --opt rmsproptf --opt-eps .001 -j 7 --warmup-lr 1e-6 --weight-decay 1e-5 --drop 0.2 --model-ema --model-ema-decay 0.9999 --aa rand-m9-mstd0.5 --remode pixel --reprob 0.2 --lr .064 --lr-noise 0.42 0.9 --amp --data-backends dali-gpu