OpenCV (v1.6)-ZHENWU PPU Cloud Service(ppu)-阿里云帮助中心

1. Overview

The PPU SDK is compatible with OpenCV's hardware acceleration capabilities. The instructions in this guide have been verified with OpenCV 4.10.

A hardware-accelerated opencv-python package for OpenCV 4.10, opencv_contrib_python-4.10.0.84, is available in the SAIL pip repository. You can install it directly:

pip install opencv-contrib-python==4.10.0.84

This guide explains how to build custom OpenCV and opencv-python packages with hardware acceleration from source.

Note

Before installing or building a custom hardware-accelerated version of OpenCV and opencv-python, uninstall any existing versions from your environment.

2. Build and install OpenCV

2.1 Download the source code

opencv_build.sh

    git clone --branch 4.10.0 --depth 1 https://github.com/opencv/opencv.git
    git clone --branch 4.10.0 --depth 1 https://github.com/opencv/opencv_contrib.git
    git clone --branch 4.10.0 --depth 1 https://github.com/opencv/opencv_extra.git
    
    cd opencv
    ln -sf ../opencv_contrib opencv_contrib

2.2 Prepare the build script

Before you start the build, ensure that you have installed the PPU SDK and configured the SDK environment by running the envsetup.sh script.

The following script disables some modules and enables as many CUDA-accelerated modules as possible. You can customize these options based on your requirements:

build_opencv.sh

mkdir build
cd build

cmake_option=""

# Configure the modules to skip. The modules in this list are examples. You can modify the list as needed.
SKIP_LIST=(xfeatures2d matlab xobjdetect xphoto wechat_qrcode)  
SKIP_DEFINES=''
for m in ${SKIP_LIST[@]}; do
    SKIP_DEFINES+=" -DBUILD_opencv_${m}=OFF"
done
cmake_option+=" ${SKIP_DEFINES}"

# Add the modules you need. This example enables as many CUDA-accelerated modules as possible.
ENABLE_LIST=(legacy ximgproc cudev cudastereo cudaarithm cudaimgproc cudacodec cudafilters cudawarping cudaoptflow cudabgsegm cudafeatures2d cudaobjdetect cudalegacy)
ENABLE_DEFINES=''
for m in ${ENABLE_LIST[@]}; do
    ENABLE_DEFINES+=" -DBUILD_opencv_${m}=ON"
done
cmake_option+=" -DWITH_CUDA=ON -DWITH_CUDNN=ON -DWITH_CUBLAS=ON -DWITH_CUFFT=ON -DOPENCV_DNN_CUDA=ON -DCUDA_ARCH_BIN=80 -DWITH_NVCUVID=ON"
cmake_option+=" -DOPENCV_ENABLE_NONFREE=ON"
cmake_option+=" ${ENABLE_DEFINES}"

# The following options disable non-essential features. You can remove these lines to enable them.
cmake_option+=" -DWITH_OPENCL=OFF -DOPENCV_DNN_OPENCL=OFF"
cmake_option+=" -DWITH_ANDROID_MEDIANDK=OFF -DWITH_GTK=OFF -DWITH_IPP=OFF -DBUILD_JAVA=OFF"
cmake_option+=" -DWITH_V4L=OFF -DWITH_GSTREAMER=OFF -DWITH_1394=OFF"

# Specify the location of the opencv_contrib modules. This path assumes the opencv_contrib directory is a sibling of the opencv directory.
cmake_option+=" -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib/modules/"

# Specify the installation directory. This sets the installation prefix to the parent directory of the source folder. If not specified, make install installs to /usr/local by default.
cmake_option+="-DOPENCV_GENERATE_PKGCONFIG -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX=../"

2.3 Configure FFmpeg support

If FFmpeg is not installed on your system, the -DWITH_FFMPEG=ON option alone is insufficient. The build will fail to locate FFmpeg, and the resulting libopencv_videoio.so library will not include FFmpeg support.

For details about building and using FFmpeg, see Video FFmpeg User Guide (v1.6).

Assuming your FFmpeg installation directory is FFMPEG_PATH, add the following lines to your build_opencv.sh script:

export PKG_CONFIG_LIBDIR=$FFMPEG_PATH/lib/pkgconfig:$PKG_CONFIG_LIBDIR
cmake_option+=" -DWITH_FFMPEG=ON -DOPENCV_FFMPEG_USE_FIND_PACKAGE=ON -DOPENCV_FFMPEG_SKIP_BUILD_CHECK=ON -DFFMPEG_DIR=./"

If the configuration is successful, you will see the following output during the build process. The version numbers after 'YES' reflect your local FFmpeg installation:

--   Video I/O:
--     FFMPEG:                      YES (find_package)
--       avcodec:                   YES (60.31.102)
--       avformat:                  YES (60.16.100)
--       avutil:                    YES (58.29.100)
--       swscale:                   YES (7.5.100)
--       avresample:                NO

Ensure the FFmpeg directory path points to the installation directory of your custom-built hardware-accelerated FFmpeg.

2.4 Configure test cases and samples

To build and install the test cases and test data included with OpenCV, add the following options. This step is optional:

cmake_option+=" -DBUILD_PERF_TESTS=ON -DBUILD_TESTS=ON -DINSTALL_TESTS=ON -DOPENCV_TEST_DATA_PATH=../opencv_extra/testdata"

You downloaded the opencv_extra directory, which contains the test data, from GitHub in a previous step.

2.5 Build

build_opencv.sh

cmake $cmake_option ..
make -j && make install

2.6 Run the examples

Change to the bin directory within your installation directory:

cd ../bin
export LD_LIBRARY_PATH=$FFMPEG_PATH/lib:../lib:$LD_LIBRARY_PATH  # Add the FFmpeg and OpenCV library directories to LD_LIBRARY_PATH
export OPENCV_TEST_DATA_PATH=../share/opencv4/testdata  # The testdata directory is from the git clone https://github.com/opencv/opencv_extra.git command

You can now run the native test applications.

3. Build and install opencv-python

3.1 Download the source code

git clone --branch 84 --depth 1 https://github.com/opencv/opencv-python.git

Download the version of opencv-python that corresponds to the OpenCV 4.10 version you built in the previous section.

3.2 Modify setup.py

Reuse the opencv and opencv_contrib source code from section 2.1; you do not need to download it again.

cd opencv-python
rm opencv opencv_contrib opencv_extra multibuild -r
# This assumes the opencv-python, opencv, and opencv_contrib directories are in the same parent directory.
ln -sf ../opencv opencv
ln -sf ../opencv_contrib opencv_contrib
ln -sf ../opencv_extra opencv_extra
git clone https://github.com/multi-build/multibuild.git

Modify setup.py to prevent it from updating the Git submodules and to change the build options. The changes are similar to those in section 2.2 and can be customized as needed:

-    if os.path.exists(".git"):
-        import pip._internal.vcs.git as git
-        g = git.Git()  # NOTE: pip API's are internal, this has to be refactored
-        g.run_command(["submodule", "sync"])
-        if build_rolling:
-            g.run_command(
-                ["submodule", "update", "--init", "--recursive", "--remote", cmake_source_dir]
-            )
-            if build_contrib:
-                g.run_command(
-                    ["submodule", "update", "--init", "--recursive", "--remote", "opencv_contrib"]
-                )
-        else:
-            g.run_command(
-                ["submodule", "update", "--init", "--recursive", cmake_source_dir]
-            )
-
-            if build_contrib:
-                g.run_command(
-                    ["submodule", "update", "--init", "--recursive", "opencv_contrib"]
-                )

... ...

             "-DBUILD_OPENEXR=ON",
+            "-DWITH_OPENCL=OFF",
+            "-DOPENCV_DNN_OPENCL=OFF",
+            "-DWITH_FFMPEG=ON",
+            "-DOPENCV_FFMPEG_USE_FIND_PACKAGE=ON",
+            "-DOPENCV_FFMPEG_SKIP_BUILD_CHECK=ON",
+            "-DFFMPEG_DIR=/usr/local", # Replace this with the installation directory of your custom-built hardware-accelerated FFmpeg.
+            "-DWITH_CUDA=ON",
+            "-DWITH_CUDNN=ON",
+            "-DWITH_CUBLAS=ON",
+            "-DWITH_CUFFT=ON",
+            "-DOPENCV_DNN_CUDA=ON",
+            "-DCUDA_ARCH_BIN=80",
+            "-DWITH_V4L=OFF",
+            "-DWITH_GSTREAMER=OFF",
+            "-DWITH_1394=OFF",
+            "-DWITH_ANDROID_MEDIANDK=OFF",
+            "-DWITH_GTK=OFF",
+            "-DWITH_IPP=OFF",
+            "-DBUILD_JAVA=OFF",
+            "-DWITH_VTK=OFF",
+            "-DENABLE_FLAKE8=OFF",
+            "-DENABLE_PYLINT=OFF",
+            "-DBUILD_opencv_legacy=ON",
+            "-DOPENCV_ENABLE_NONFREE=ON",
+            "-DBUILD_opencv_xfeatures2d=OFF",
+            "-DBUILD_opencv_matlab=OFF",
+            "-DBUILD_opencv_xobjdetect=OFF",
+            "-DBUILD_opencv_xphoto=OFF",
+            "-DBUILD_opencv_wechat_qrcode=OFF",
+            "-DBUILD_opencv_ximgproc=ON",
+            "-DBUILD_opencv_cudev=ON",
+            "-DBUILD_opencv_cudastereo=ON",
+            "-DBUILD_opencv_cudaarithm=ON",
+            "-DBUILD_opencv_cudaimgproc=ON",
+            "-DBUILD_opencv_cudacodec=ON",
+            "-DBUILD_opencv_cudafilters=ON",
+            "-DBUILD_opencv_cudawarping=ON",
+            "-DBUILD_opencv_cudaoptflow=ON",
+            "-DBUILD_opencv_cudabgsegm=ON",
+            "-DBUILD_opencv_cudafeatures2d=ON",
+            "-DBUILD_opencv_cudaobjdetect=ON",
+            "-DBUILD_opencv_cudalegacy=ON",
+            "-DOPENCV_GENERATE_PKGCONFIG=ON"
        ]

3.3 Build the package

This process requires scikit-build. You can install it by running pip install scikit-build.

export ENABLE_CONTRIB=1
python3 setup.py bdist_wheel

3.4 Install the package

The build process generates the opencv_contrib_python-4.10.0.84-cp37-cp37m-linux_x86_64.whl file in the dist directory.

pip install dist/opencv_contrib_python-4.10.0.84-cp37-cp37m-linux_x86_64.whl

At this point, you can enter the Python interactive shell and type import cv2 to check if the import is successful.

4. Known issues

cuBLAS:
- GEMM: cublasCgemm_v2 is not supported.
cuFFT:
- Dft/Convolve: cufftPlan2d is not supported.
cudacodec:
- Older video formats such as MPEG4, MPEG2, VC1, and VP8 are not supported.
cudaoptflow
- Some test items are not supported because the PPU does not have opt flow hardware.

In addition, a few native test items fail on the PPU. Analysis shows that these failures can be skipped. The main reasons are as follows:

Inconsistent results occur because of minor errors in floating-point precision. These errors are within the acceptable unit in the last place (ULP) range. Some of these test items also fail on NVIDIA A100 cards.
Results differ because kernel code variables are not initialized to 0, or because allocated GPU memory is not set to 0 using memset. In the NVIDIA environment, local variables and allocated device memory are initialized to 0 by default. The PPU does not do this.
The use of different operators or algorithms causes different results.

5. Example

This example shows how to use opencv-python for hardware codec acceleration. This speeds up the preprocessing and post-processing for object detection model inference with Ultralytics YOLOv5.

https://github.com/ultralytics/yolov5.git

To enable hardware-accelerated PPU Video decoding for preprocessing, explicitly use FFmpeg for VideoCapture and configure the cuvid hardware acceleration options. See the following code snippet:

object_tracking.py

+os.environ["OPENCV_FFMPEG_CAPTURE_OPTIONS"] = "hwaccel;cuvid|video_codec;h264_cuvid|vsync;0"
-cap = cv2.VideoCapture(path)
+cap = cv2.VideoCapture(path, cv2.CAP_FFMPEG)

For post-processing, cv2.VideoWriter uses FFmpeg for video encoding by default. If you configure FFmpeg with hardware acceleration as described in this document, hardware encoding is used by default. No code changes are necessary:

object_tracking.py

result = cv2.VideoWriter("object_tracking.mp4",
                       cv2.VideoWriter_fourcc(*'avc1'),
                       fps,
                       (w, h))