DALI (v1.6)-ZHENWU PPU Cloud Service(ppu)-阿里云帮助中心

1. Overview

NVIDIA DALI is an open-source framework that provides a parallel data loader for model training and supports hardware encoding and decoding acceleration. The T-Head SAIL SDK supports the DALI framework, which significantly accelerates model training performance in CPU-bound scenarios.

Official website: https://docs.nvidia.com/deeplearning/dali/user-guide/docs/index.html

Source code: https://github.com/NVIDIA/DALI

2. Install DALI

2.1 Docker

If you use the Docker image provided with the SAIL SDK, the preinstalled DALI is ready to use. The default version of DALI in the Docker image is v1.20:

pip list | grep nvidia-dali

2.2 Install from pip source

You can install DALI by using a whl package from the T-Head pip source. The SAIL SDK v1.4.1 release adds support for DALI v1.44. Select and install the whl package that matches your DALI version (v1.20 or v1.44), operating system (Ubuntu or AliOS), and CUDA version.

For example, to install DALI v1.20 on Ubuntu with CUDA 11.6, run the following command:

pip install nvidia_dali_cuda110==1.20.0+ppu202412182301.ce -i https://art-pub.eng.t-head.cn/artifactory/api/pypi/ptgai-pypi_ppu_ubuntu_cu116_index/simple/
# ppu2024xxxxxxxx is the version number for the SAIL SDK v1.4.1 release.

To install DALI v1.44 on Ubuntu with CUDA 12.4, run the following command:

pip install nvidia_dali_cuda110==1.44.0+ppu202412182301.ce -i https://art-pub.eng.t-head.cn/artifactory/api/pypi/ptgai-pypi_ppu_ubuntu_cu124_index/simple/
# ppu2024xxxxxxxx is the version number for the SAIL SDK v1.4.1 release.

Note

DALI v1.44 depends on the NVIDIA nvImageCodec project: https://github.com/NVIDIA/nvImageCodec. The minimum required CUDA version for this project is CUDA 11.8. Therefore, you must use DALI v1.44 in an environment with CUDA 11.8 or later.
Apart from the previous limitation, DALI depends only on the major CUDA version. Therefore, a DALI whl package for CUDA 12.4 works with CUDA Toolkit 12.1 or 12.6 without requiring a reinstallation. For more information, see the official NVIDIA documentation: https://docs.nvidia.com/deeplearning/dali/user-guide/docs/support_matrix.html.
Starting from the SAIL SDK v1.4.1 release, DALI supports the video loader feature, which provides hardware-accelerated video decoding. To use the video loader, we recommend DALI v1.44 because the official support for the video loader feature in v1.20 is incomplete. The following is a non-exhaustive list of features and fixes for the video loader in DALI v1.44 compared to DALI v1.20:
1. Improved robustness of VideoReader and enhanced handling of error streams.
2. Fixed an infinite loop issue in the video decoder seek.
3. Added support for video streams with different bit depths.

2.3 Download NVIDIA WHL files

Not recommended. You cannot directly use the DALI installation packages provided by NVIDIA.

2.4 Compile from source

Download and compile DALI dependencies:

git clone --depth 1 --branch v1.44.0 --recurse-submodules https://github.com/NVIDIA/DALI_deps.git /tmp/DALI_deps
cd /tmp/DALI_deps
bash build_scripts/build_deps.sh

Download and compile DALI:

git clone --branch release_v1.44 --recurse-submodules https://github.com/NVIDIA/DALI.git
cd /workspace/DALI
git apply t-head_dali.patch # !!!A patch is required
mkdir build && cd build
cmake .. -DBUILD_BENCHMARK=OFF -DWITH_DYNAMIC_NVIMGCODEC=OFF -DBUILD_CVCUDA=OFF # Will fix CVCUDA compile issue soon
make -j

The main content of the patch is as follows:

In third_party/cutlass/include/cutlass/arch/wmma.h:

-#if !(defined(__clang__) && defined(__CUDA__))
+#if !(defined(__clang__) && defined(__CUDA__)) && !defined(__HGGCCC__)

In CMakeLists.txt:

-set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} --compiler-options \"-fvisibility=hidden -Wno-free-nonheap-object\" --Wno-deprecated-gpu-targets -Xfatbin -compress-all")
+set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} --compiler-options \"-fvisibility=hidden -Wno-free-nonheap-object -fno-delayed-template-parsing\" --Wno-deprecated-gpu-targets -Xfatbin -compress-all")

3. Enable DALI

You can refer to the official NVIDIA documentation for implementing code to enable DALI from scratch:

https://docs.nvidia.com/deeplearning/dali/user-guide/docs/examples/getting_started.html

To enable DALI in some deep learning frameworks, refer to the following code examples:

4. DALI in open-source libraries

4.1 DLE

Some models in NVIDIA Deep Learning Examples (DLE) natively support the DALI dataloader.

GitHub - NVIDIA/DeepLearningExamples: State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.

Specify dali-gpu as the data backend in the run command.

Take ResNet50 training as an example:

python ./multiproc.py --nproc_per_node 8 ./main.py --arch resnet50 -c fanin --label-smoothing 0.1 --data-backends dali-gpu <path to imagenet>

4.2 PyTorch image models

PyTorch Image Models (timm): https://github.com/huggingface/pytorch-image-models

Take MobileNetV3 training as an example:

bash ./distributed_train.sh 2 /root/datasets/vision/imagenet --model mobilenetv3_large_100 -b 512 --sched step --epochs 1  --decay-epochs 2.4 --decay-rate .973 --opt rmsproptf --opt-eps .001 -j 7 --warmup-lr 1e-6 --weight-decay 1e-5 --drop 0.2 --model-ema --model-ema-decay 0.9999 --aa rand-m9-mstd0.5 --remode pixel --reprob 0.2 --lr .064 --lr-noise 0.42 0.9 --amp --data-backends dali-gpu