DALI (v1.4.1)
1. Overview
NVIDIA DALI is an open-source framework that provides parallel data loaders for model training and hardware-accelerated encoding and decoding. The T-Head SAIL SDK supports the DALI framework, which significantly accelerates model training in CPU-bound scenarios.
Official website: https://docs.nvidia.com/deeplearning/dali/user-guide/docs/index.html
Source code: https://github.com/NVIDIA/DALI
2. Installation
2.1 Docker
The T-Head SAIL SDK Docker image comes with DALI v1.20 pre-installed.
pip list | grep nvidia-dali2.2 From pip source
To use DALI, install the whl package from the T-Head pip source. The SAIL SDK v1.4.1 release adds support for DALI v1.44. Select the whl package that matches your DALI version (v1.20 or v1.44), OS (Ubuntu or AliOS), and CUDA version.
For example, to install DALI v1.20 on Ubuntu with CUDA 11.6, run the following command:
pip install nvidia_dali_cuda110==1.20.0+ppu202412182301.ce -i https://art-pub.eng.t-head.cn/artifactory/api/pypi/ptgai-pypi_ppu_ubuntu_cu116_index/simple/
# ppu2024xxxxxxxx is the T-Head SAIL SDK version number for the v1.4.1 releaseTo install DALI v1.44 on Ubuntu with CUDA 12.4, run the following command:
pip install nvidia_dali_cuda110==1.44.0+ppu202412182301.ce -i https://art-pub.eng.t-head.cn/artifactory/api/pypi/ptgai-pypi_ppu_ubuntu_cu124_index/simple/
# ppu2024xxxxxxxx is the T-Head SAIL SDK version number for the v1.4.1 releaseDALI v1.44 depends on the NVIDIA nvImageCodec project: https://github.com/NVIDIA/nvImageCodec. This project requires a minimum CUDA version of 11.8. Therefore, DALI v1.44 requires an environment with CUDA 11.8 or later.
DALI depends only on the major CUDA version. For example, a whl package for CUDA 12.4 is compatible with any CUDA 12.x CUDA Toolkit, such as v12.1 or v12.6, and does not require reinstallation. For more information, refer to the official NVIDIA documentation: https://docs.nvidia.com/deeplearning/dali/user-guide/docs/support_matrix.html.
SAIL SDK v1.4.1 adds support for the video loader feature, which provides hardware accelerated video decoding. DALI v1.44 is recommended if you need to use this feature, as NVIDIA made significant improvements in v1.44 compared to the less complete support in v1.20. The video loader in DALI v1.44 includes the following features and fixes:
Enhanced VideoReader robustness with improved handling of a corrupted stream.
Fixed an infinite loop issue during seek operations in the video decoder.
Added support for video streams with different bit depths.
2.3 Direct download from NVIDIA
This method is not recommended because the official DALI installation packages from NVIDIA are not directly compatible.
2.4 Build from source
Download and build the DALI dependencies:
git clone --depth 1 --branch v1.44.0 --recurse-submodules https://github.com/NVIDIA/DALI_deps.git /tmp/DALI_deps
cd /tmp/DALI_deps
bash build_scripts/build_deps.shDownload and build DALI:
git clone --branch release_v1.44 --recurse-submodules https://github.com/NVIDIA/DALI.git
cd /workspace/DALI
git apply t-head_dali.patch # A patch is required!
mkdir build && cd build
cmake .. -DBUILD_BENCHMARK=OFF -DWITH_DYNAMIC_NVIMGCODEC=OFF -DBUILD_CVCUDA=OFF # CVCUDA is disabled due to a pending compile fix.
make -jThe patch includes the following changes:
third_party/cutlass/include/cutlass/arch/wmma.h
-#if !(defined(__clang__) && defined(__CUDA__))
+#if !(defined(__clang__) && defined(__CUDA__)) && !defined(__HGGCCC__)CMakeLists.txt
-set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} --compiler-options \"-fvisibility=hidden -Wno-free-nonheap-object\" --Wno-deprecated-gpu-targets -Xfatbin -compress-all")
+set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} --compiler-options \"-fvisibility=hidden -Wno-free-nonheap-object -fno-delayed-template-parsing\" --Wno-deprecated-gpu-targets -Xfatbin -compress-all")3. Enabling DALI
To implement DALI from scratch, refer to the official NVIDIA documentation:
https://docs.nvidia.com/deeplearning/dali/user-guide/docs/examples/getting_started.html
For examples of enabling DALI in specific deep learning frameworks, see the following code samples:
PyTorch: https://github.com/NVIDIA/DALI/blob/main/docs/examples/use_cases/pytorch/resnet50/main.py
PaddlePaddle: https://github.com/NVIDIA/DALI/blob/main/docs/examples/use_cases/paddle/resnet50/dali.py
4. DALI support in open source libraries
4.1 DLE
Some models in NVIDIA Deep Learning Examples (DLE) natively support the DALI data loader.
https://github.com/NVIDIA/DeepLearningExamples
Simply specify dali-gpu as the data backend in the run command.
For example, to train ResNet50:
python ./multiproc.py --nproc_per_node 8 ./main.py --arch resnet50 -c fanin --label-smoothing 0.1 --data-backends dali-gpu <path to imagenet>4.2 PyTorch image models
PyTorch image models (timm): https://github.com/huggingface/pytorch-image-models
The following command trains a MobileNetV3 model:
bash ./distributed_train.sh 2 /root/datasets/vision/imagenet --model mobilenetv3_large_100 -b 512 --sched step --epochs 1 --decay-epochs 2.4 --decay-rate .973 --opt rmsproptf --opt-eps .001 -j 7 --warmup-lr 1e-6 --weight-decay 1e-5 --drop 0.2 --model-ema --model-ema-decay 0.9999 --aa rand-m9-mstd0.5 --remode pixel --reprob 0.2 --lr .064 --lr-noise 0.42 0.9 --amp --data-backends dali-gpu