DocsICP FilingConsole
Documentation
Home Page ZHENWU PPU Cloud Service Developer Reference PPU SDK v1.7.x DALI user guide (v1.7)

DALI user guide (v1.7)

Updated at:
Copy as MD
My Favorites

1. Overview

NVIDIA DALI is an open-source library that provides a parallel data loader to accelerate model training. It supports hardware-accelerated encoding/decoding. The T-Head SAIL SDK integrates with DALI to significantly improve training performance, especially in CPU-bound scenarios.

Official website: https://docs.nvidia.com/deeplearning/dali/user-guide/docs/index.html

Source code: https://github.com/NVIDIA/DALI

2. Installing DALI

2.1 Docker

The SAIL SDK Docker image comes with DALI v1.20 pre-installed. 

pip list | grep nvidia-dali

2.2 Installing from a pip source

You can install DALI from the T-Head pip source. SAIL SDK v1.4.1 adds support for DALI v1.44. Choose the whl package that matches your DALI version (v1.20 or v1.44), OS (Ubuntu or AliOS), and CUDA version.

For example, to install DALI v1.20 on Ubuntu with CUDA 11.6, run the following command: 

pip install nvidia_dali_cuda110==1.20.0+ppu202412182301.ce -i https://art-pub.eng.t-head.cn/artifactory/api/pypi/ptgai-pypi_ppu_ubuntu_cu116_index/simple/
# ppu2024xxxxxxxx is the version number of the SAIL SDK v1.4.1 release.

To install DALI v1.44 on Ubuntu with CUDA 12.4, run the following command: 

pip install nvidia_dali_cuda110==1.44.0+ppu202412182301.ce -i https://art-pub.eng.t-head.cn/artifactory/api/pypi/ptgai-pypi_ppu_ubuntu_cu124_index/simple/
# ppu2024xxxxxxxx is the version number of the SAIL SDK v1.4.1 release.
Note

  1. DALI v1.44 depends on the NVIDIA nvImageCodec project: https://github.com/NVIDIA/nvImageCodec. This project requires CUDA 11.8 or later. Therefore, you must use DALI v1.44 in an environment with CUDA 11.8 or later.

  2. Except for the limitation mentioned in Note 1, DALI depends only on the major version of CUDA. This means that if you have installed the DALI wheel package for CUDA 12.4, you do not need to reinstall DALI if you upgrade the CUDA Toolkit to CUDA 12.6 or downgrade it to CUDA 12.1. For more information, see the official NVIDIA documentation:https://docs.nvidia.com/deeplearning/dali/user-guide/docs/support_matrix.html.

  3. Starting with SAIL SDK v1.4.1, DALI supports the video loader feature, which provides video hardware decoding acceleration. If you plan to use the video loader, we recommend DALI v1.44 because its support for this feature is more complete than in v1.20. DALI v1.44 includes the following improvements to the video loader:

    1. Improved VideoReader robustness for handling malformed streams.

    2. Fixed an infinite loop issue in video decoder seek operations.

    3. Added support for video streams with different bit depths.

2.3 Using official whl packages from NVIDIA

Not recommended. You cannot directly use the DALI installation packages provided by NVIDIA.

2.4 Building from source code

Download and build the DALI dependencies: 

git clone --depth 1 --branch v1.44.0 --recurse-submodules https://github.com/NVIDIA/DALI_deps.git /tmp/DALI_deps
cd /tmp/DALI_deps
bash build_scripts/build_deps.sh

Download and build DALI:

git clone --branch release_v1.44 --recurse-submodules https://github.com/NVIDIA/DALI.git
cd /workspace/DALI
git apply t-head_dali.patch # !!! Patch required
mkdir build && cd build
cmake .. -DBUILD_BENCHMARK=OFF -DWITH_DYNAMIC_NVIMGCODEC=OFF -DBUILD_CVCUDA=OFF # CVCUDA build is disabled due to a compilation issue.
make -j

The patch includes the following main changes:

In third_party/cutlass/include/cutlass/arch/wmma.h:

-#if !(defined(__clang__) && defined(__CUDA__))
+#if !(defined(__clang__) && defined(__CUDA__)) && !defined(__HGGCCC__)

In CMakeLists.txt:

-set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} --compiler-options \"-fvisibility=hidden -Wno-free-nonheap-object\" --Wno-deprecated-gpu-targets -Xfatbin -compress-all")
+set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} --compiler-options \"-fvisibility=hidden -Wno-free-nonheap-object -fno-delayed-template-parsing\" --Wno-deprecated-gpu-targets -Xfatbin -compress-all")

3. Enabling DALI

To enable DALI from scratch, refer to the official NVIDIA getting started guide:

https://docs.nvidia.com/deeplearning/dali/user-guide/docs/examples/getting_started.html

For integration with specific deep learning frameworks, see the following reference implementations:

  1. PyTorch: https://github.com/NVIDIA/DALI/blob/main/docs/examples/use_cases/pytorch/resnet50/main.py

  2. TensorFlow: https://github.com/NVIDIA/DALI/blob/main/docs/examples/use_cases/tensorflow/resnet-n/nvutils/image_processing.py

  3. PaddlePaddle: https://github.com/NVIDIA/DALI/blob/main/docs/examples/use_cases/paddle/resnet50/dali.py

4. DALI support in open-source libraries

4.1 Nvidia deep learning examples (DLE)

Some models in the Nvidia Deep Learning Examples (DLE) natively support the DALI data loader.

GitHub - NVIDIA/DeepLearningExamples: State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.

Simply specify dali-gpu as the data backend in the run command.

Example: Train ResNet50 

python ./multiproc.py --nproc_per_node 8 ./main.py --arch resnet50 -c fanin --label-smoothing 0.1 --data-backends dali-gpu <path to imagenet>

4.2 PyTorch image models

PyTorch Image Models (timm): https://github.com/huggingface/pytorch-image-models

Example: Train mobilenetv3 

bash ./distributed_train.sh 2 /root/datasets/vision/imagenet --model mobilenetv3_large_100 -b 512 --sched step --epochs 1  --decay-epochs 2.4 --decay-rate .973 --opt rmsproptf --opt-eps .001 -j 7 --warmup-lr 1e-6 --weight-decay 1e-5 --drop 0.2 --model-ema --model-ema-decay 0.9999 --aa rand-m9-mstd0.5 --remode pixel --reprob 0.2 --lr .064 --lr-noise 0.42 0.9 --amp --data-backends dali-gpu
Previous：Video Codec/Image (v1.7)Next：Open3D (v1.7)