DocsICP FilingConsole
Documentation
Home Page ZHENWU PPU Cloud Service Developer Reference PPU SDK v1.7.x OpenCV usage (v1.7)

OpenCV usage (v1.7)

Updated at:
Copy as MD
My Favorites

1. Overview

The PPU SDK is compatible with OpenCV's hardware acceleration features. The currently verified OpenCV version is 4.10.

The SAIL pip repository provides the opencv_contrib_python-4.10.0.84 package, a hardware-accelerated opencv-python library based on OpenCV 4.10. You can install it directly by running the following command:

pip install opencv-contrib-python==4.10.0.84

This guide describes how to compile your own hardware-accelerated OpenCV and opencv-python packages from source.

Note

Before installing or building these packages, first uninstall any existing OpenCV and opencv-python packages from your environment.

2. Compile and install OpenCV

2.1 Download source code

Run the following commands to download the source code:

    git clone --branch 4.10.0 --depth 1 https://github.com/opencv/opencv.git
    git clone --branch 4.10.0 --depth 1 https://github.com/opencv/opencv_contrib.git
    git clone --branch 4.10.0 --depth 1 https://github.com/opencv/opencv_extra.git
    
    cd opencv
    ln -sf ../opencv_contrib opencv_contrib

2.2 Prepare build script

Before you compile, install the PPU SDK and run envsetup.sh to configure the SDK environment.

The following script removes some modules and enables as many CUDA-accelerated modules as possible. You can customize these settings as needed.

Create a file named build_opencv.sh with the following content:

mkdir build
cd build

cmake_option=""

# Configure modules to skip. This list is just an example; modify it as needed.
SKIP_LIST=(xfeatures2d matlab xobjdetect xphoto wechat_qrcode)  
SKIP_DEFINES=''
for m in ${SKIP_LIST[@]}; do
    SKIP_DEFINES+=" -DBUILD_opencv_${m}=OFF"
done
cmake_option+=" ${SKIP_DEFINES}"

# Add required modules. This configuration enables as many CUDA-accelerated modules as possible.
ENABLE_LIST=(legacy ximgproc cudev cudastereo cudaarithm cudaimgproc cudacodec cudafilters cudawarping cudaoptflow cudabgsegm cudafeatures2d cudaobjdetect cudalegacy)
ENABLE_DEFINES=''
for m in ${ENABLE_LIST[@]}; do
    ENABLE_DEFINES+=" -DBUILD_opencv_${m}=ON"
done
cmake_option+=" -DWITH_CUDA=ON -DWITH_CUDNN=ON -DWITH_CUBLAS=ON -DWITH_CUFFT=ON -DOPENCV_DNN_CUDA=ON -DCUDA_ARCH_BIN=80 -DWITH_NVCUVID=ON"
cmake_option+=" -DOPENCV_ENABLE_NONFREE=ON"
cmake_option+=" ${ENABLE_DEFINES}"

# Disable some non-essential options. You can re-enable them if needed.
cmake_option+=" -DWITH_OPENCL=OFF -DOPENCV_DNN_OPENCL=OFF"
cmake_option+=" -DWITH_ANDROID_MEDIANDK=OFF -DWITH_GTK=OFF -DWITH_IPP=OFF -DBUILD_JAVA=OFF"
cmake_option+=" -DWITH_V4L=OFF -DWITH_GSTREAMER=OFF -DWITH_1394=OFF"

# Specify the path to the extra modules. This assumes opencv_contrib is a sibling of the opencv directory.
cmake_option+=" -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib/modules/"

# Specify the installation prefix. This example installs to the parent directory of the source code. If not specified, 'make install' installs to /usr/local.
cmake_option+="-DOPENCV_GENERATE_PKGCONFIG -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX=../"

2.3 Configure FFmpeg support

If FFmpeg is not installed on the system, the compiler will fail to find FFmpeg during the OpenCV configuration process even if you only enable the -DWITH_FFMPEG=ON flag. As a result, the compiled libopencv_videoio.so will not include support for FFmpeg.

Refer to the Video FFmpeg User Guide (v1.6) for instructions on how to compile and use FFmpeg.

Assuming your FFmpeg installation directory is FFMPEG_PATH, add the following lines to your build_opencv.sh script:

export PKG_CONFIG_LIBDIR=$FFMPEG_PATH/lib/pkgconfig:$PKG_CONFIG_LIBDIR
cmake_option+=" -DWITH_FFMPEG=ON -DOPENCV_FFMPEG_USE_FIND_PACKAGE=ON -DOPENCV_FFMPEG_SKIP_BUILD_CHECK=ON -DFFMPEG_DIR=./"

If the configuration is successful, the build produces output similar to the following. The version numbers shown will reflect your local FFmpeg version. 

--   Video I/O:
--     FFMPEG:                      YES (find_package)
--       avcodec:                   YES (60.31.102)
--       avformat:                  YES (60.16.100)
--       avutil:                    YES (58.29.100)
--       swscale:                   YES (7.5.100)
--       avresample:                NO

Point the FFmpeg directory to the installation directory of your self-compiled, hardware-accelerated FFmpeg.

2.4 Configure tests and samples

To compile and install the built-in test cases and test data for OpenCV, add the following options to cmake_option. This step is optional. 

cmake_option+=" -DBUILD_PERF_TESTS=ON -DBUILD_TESTS=ON -DINSTALL_TESTS=ON -DOPENCV_TEST_DATA_PATH=../opencv_extra/testdata"

Here, opencv_extra is the directory containing the test data, cloned from GitHub in a previous step.

2.5 Compile

Add the following lines to the end of your build_opencv.sh script and then run it:

cmake $cmake_option ..
make -j && make install

2.6 Run sample

Navigate to the bin directory within your installation path:

cd ../bin
# Add the FFmpeg and OpenCV library directories to LD_LIBRARY_PATH.
export LD_LIBRARY_PATH=$FFMPEG_PATH/lib:../lib:$LD_LIBRARY_PATH
# The testdata directory is obtained by running git clone https://github.com/opencv/opencv_extra.git.
export OPENCV_TEST_DATA_PATH=../share/opencv4/testdata

You can now run the native test programs.

3. Compile and install opencv-python

3.1 Download source code

git clone --branch 84 --depth 1 https://github.com/opencv/opencv-python.git

Since you compiled OpenCV 4.10 in the previous section, you must download the corresponding version of opencv-python.

3.2 Modify setup.py

You can reuse the opencv and opencv_contrib source code downloaded in Section 2.1 instead of downloading them again.

cd opencv-python
rm opencv opencv_contrib opencv_extra multibuild -r
# This assumes that opencv-python, opencv, and opencv_contrib are in the same parent directory.
ln -sf ../opencv opencv
ln -sf ../opencv_contrib opencv_contrib
ln -sf ../opencv_extra opencv_extra
git clone https://github.com/multi-build/multibuild.git

Modify setup.py to prevent it from running git submodule update and to change the compile options. The required modifications are similar to those in Section 2.2 and can be customized as needed.

-    if os.path.exists(".git"):
-        import pip._internal.vcs.git as git
-        g = git.Git()  # NOTE: pip API's are internal, this has to be refactored
-        g.run_command(["submodule", "sync"])
-        if build_rolling:
-            g.run_command(
-                ["submodule", "update", "--init", "--recursive", "--remote", cmake_source_dir]
-            )
-            if build_contrib:
-                g.run_command(
-                    ["submodule", "update", "--init", "--recursive", "--remote", "opencv_contrib"]
-                )
-        else:
-            g.run_command(
-                ["submodule", "update", "--init", "--recursive", cmake_source_dir]
-            )
-
-            if build_contrib:
-                g.run_command(
-                    ["submodule", "update", "--init", "--recursive", "opencv_contrib"]
-                )

... ...

             "-DBUILD_OPENEXR=ON",
+            "-DWITH_OPENCL=OFF",
+            "-DOPENCV_DNN_OPENCL=OFF",
+            "-DWITH_FFMPEG=ON",
+            "-DOPENCV_FFMPEG_USE_FIND_PACKAGE=ON",
+            "-DOPENCV_FFMPEG_SKIP_BUILD_CHECK=ON",
+            "-DFFMPEG_DIR=/usr/local", # Replace this with the installation directory of your self-compiled, hardware-accelerated FFmpeg.
+            "-DWITH_CUDA=ON",
+            "-DWITH_CUDNN=ON",
+            "-DWITH_CUBLAS=ON",
+            "-DWITH_CUFFT=ON",
+            "-DOPENCV_DNN_CUDA=ON",
+            "-DCUDA_ARCH_BIN=80",
+            "-DWITH_V4L=OFF",
+            "-DWITH_GSTREAMER=OFF",
+            "-DWITH_1394=OFF",
+            "-DWITH_ANDROID_MEDIANDK=OFF",
+            "-DWITH_GTK=OFF",
+            "-DWITH_IPP=OFF",
+            "-DBUILD_JAVA=OFF",
+            "-DWITH_VTK=OFF",
+            "-DENABLE_FLAKE8=OFF",
+            "-DENABLE_PYLINT=OFF",
+            "-DBUILD_opencv_legacy=ON",
+            "-DOPENCV_ENABLE_NONFREE=ON",
+            "-DBUILD_opencv_xfeatures2d=OFF",
+            "-DBUILD_opencv_matlab=OFF",
+            "-DBUILD_opencv_xobjdetect=OFF",
+            "-DBUILD_opencv_xphoto=OFF",
+            "-DBUILD_opencv_wechat_qrcode=OFF",
+            "-DBUILD_opencv_ximgproc=ON",
+            "-DBUILD_opencv_cudev=ON",
+            "-DBUILD_opencv_cudastereo=ON",
+            "-DBUILD_opencv_cudaarithm=ON",
+            "-DBUILD_opencv_cudaimgproc=ON",
+            "-DBUILD_opencv_cudacodec=ON",
+            "-DBUILD_opencv_cudafilters=ON",
+            "-DBUILD_opencv_cudawarping=ON",
+            "-DBUILD_opencv_cudaoptflow=ON",
+            "-DBUILD_opencv_cudabgsegm=ON",
+            "-DBUILD_opencv_cudafeatures2d=ON",
+            "-DBUILD_opencv_cudaobjdetect=ON",
+            "-DBUILD_opencv_cudalegacy=ON",
+            "-DOPENCV_GENERATE_PKGCONFIG=ON"
        ]

3.3 Compile

This has a dependency on skbuild. You can install it by running pip install scikit-build.

export ENABLE_CONTRIB=1
python3 setup.py bdist_wheel

3.4 Install

The build process generates a wheel file, such as opencv_contrib_python-4.10.0.84-cp37-cp37m-linux_x86_64.whl, in the dist/ directory.

pip install dist/opencv_contrib_python-4.10.0.84-cp37-cp37m-linux_x86_64.whl

At this point, you can open the Python interactive shell and enter import cv2 to see if it is successful.

4. Known issues

  • cuBLAS:

    • GEMM: cublasCgemm_v2 is not supported.

  • cuFFT:

    • Dft/Convolve: cufftPlan2d is not supported.

  • cudacodec:

    • Older video formats such as MPEG-4, MPEG-2, VC-1, and VP8 are not supported.

  • cudaoptflow

    • Some test cases are not supported because the PPU does not include optical flow hardware.

In addition, a small number of native test cases may fail on the PPU. These failures can be safely ignored and are primarily due to the following reasons:

  • Slight discrepancies in floating-point precision that are within the acceptable Units in the Last Place (ULP) tolerance. Some of these same test cases also fail on NVIDIA A100 GPUs.

  • Differences in results may be caused by uninitialized variables in a kernel or by not clearing allocated device memory to zero. In NVIDIA environments, local variables and allocated device memory are often initialized to zero by default, but this is not the case on the PPU.

  • Differences in results may be caused by the use of different operators or algorithms.

5. Usage example

This example shows how to use a hardware-accelerated opencv-python package with Ultralytics YOLOv5 to accelerate video encoding and decoding for the preprocessing and postprocessing steps in an object detection task.

https://github.com/ultralytics/yolov5.git

To enable hardware-accelerated PPU video decoding for preprocessing, you must explicitly use FFmpeg for video capture and configure the cuvid hardware acceleration options. See the following code snippet:

object_tracking.py

+os.environ["OPENCV_FFMPEG_CAPTURE_OPTIONS"] = "hwaccel;cuvid|video_codec;h264_cuvid|vsync;0"
-cap = cv2.VideoCapture(path)
+cap = cv2.VideoCapture(path, cv2.CAP_FFMPEG)

For postprocessing, cv2.VideoWriter uses FFmpeg for video encoding by default. If you have configured a hardware-accelerated FFmpeg as described in this guide, hardware encoding is enabled automatically, requiring no code changes:

object_tracking.py

result = cv2.VideoWriter("object_tracking.mp4",
                       cv2.VideoWriter_fourcc(*'avc1'),
                       fps,
                       (w, h))
Previous：PPU-SMI (v1.7)Next：FFmpeg video (v1.7)