How to install Nvidia Gstreamer plugins (nvenc, nvdec) on Ubuntu?

7 min. read |

Gstreamer’s plugins avdec_h264 (h264 video decoding) and x264enc (h264 video encoding) optimized for CPU. Meanwhile with Nvidia Gstreamer plugins (nvenc, nvdec) we can benefit from GPU capabilities and performance. With GPU based plugins applications can process videos up to 3 times faster.

Requirements

Scripts

Learn how to?

Guide

Environment setup

  • CUDA Version 10.0.130
  • Ubuntu 18.04.4 LTS
  • Video_Codec_SDK_9.0.20
  • Gstreamer 1.14.5
  • NVIDIA Driver 435.21

Inspect

At the beginning, check if plugins are already installed:

On test machine plugin isn’t installed yet, so command’s output looks like the following:

Requirements

First, check your GPU supports NVENC, NVDEC SDKs using Video Encode and Decode GPU Support Matrix

Initial requirements for nvenc, nvdec plugins specified in official gstreamer nvidia plugins README.

Install CUDA

Check CUDA version:

Note: How to check CUDA and cuDNN version

Install CUDA Toolkit using official NVIDIA Cuda Installation Guide

Install NVIDIA Video Codec SDK

Download

At first, go to NVIDIA Video Codec SDK’s page and download compatible with your nvidia driver version. For example, check the requirements for the latest SDK version:

Use nvidia-smi to check your nvidia driver version.

Note: In particular case my local Driver Version is 435.21 and Nvidia Video Codec SDK 9.1 requires 435.21 or newer. But in my case I managed everything to work with previous version of Nvidia Video Codec SDK 9.0 with next requirements:

Note: Legacy versions of Video Codec SDK download page

Install

In order to install Video Codec SDK simply extract files from downloaded archive and move includes & libs to your cuda path (ex.: usr/local/cuda/). For example:

Build nvenc, nvdec gstreamer plugins

Nvidia gstreamer plugins located in gst-plugins-bad package.

Note: in gst-plugins-bad repository plugins need more quality, testing or documentation

Now, let’s clone repository and build plugins.

Note: additional NVENCODE_CFLAGS instructs to look at Cuda’s includes folder when building plugins. And flag (–with-cuda-prefix) specifies exact Cuda location.

Check output to confirm that nvenc, nvdec plugins is going to be build.

Then, go to exact nvenc, nvdec folders and build libraries.

Note: By default gstreamer installs libraries (*.so) into /usr/local/lib/gstreamer-1.0/, if no other location specified by –prefix option when configuring.

Best practice to load gstreamer plugins from other locations just specify GST_PLUGIN_PATH environment variable. For example:

Now, check that both nvenc and nvdec plugins available with gst-inspect-1.0

Or

Note: Enable error messages for nvidia gstreamer plugins to check exact reason in case of failure with the following export:

Now, let’s launch some pipelines to check correctness of Nvidia’s Gstreamer plugins video decoding/encoding

Nvidia-accelerated pipelines

Plugins Performance

Download test video with youtube-dl. Have a look at How to watch Youtube videos with Gstreamer to explore youtube-dl.

Let’s check video information with gst-discoverer-1.0:

Note: Pay attention to container type (QuickTime), container format (MP4), video codec (H264). This information helps to build right pipeline to process video.

In addition, let use ffprobe to get video’s frames count and resolution

Hardware specifications

  • GPU GeForce GTX 1050
  • CPU Intel(R) Core(TM) i7-7700HQ @ 2.80GHz

At first let’s try to decode video with avdec_h264 plugin. It is pure implementation from Libav (open-source audio and video processing library). Common pipeline looks like the following:

Note: avdec_h264 plugin is used as video codec format is H264.

Note: qtdemux used to demux video with QuickTime container format of video file

Also let check CPU-GPU load while video file processing. CPU is loaded for ~65% and GPU is in idle state.

Now, try to decode video with nvdec

Note that now GPU loaded for ~20% and there is a process gst-launch-1.0 located on GPU. CPU load also around 20%, that is less when decoding pipeline with avdec_h264 (65%).

Note: Decoding with NVIDIA plugin appeared to be slower (~2.6 s vs 4.1 s), but it possibly due to the CPU -> GPU buffers transfer and due to the initial memory allocation on GPU. For bigger (ex.: 4K) and longer video files decoding on GPU is going to be faster.

Now, check encoding performance using x264enc and nvh264enc plugins.

Decoding/Encoding pluginCPUGPUElapsed TimeFPSScaler
avdec_h264~65%0%2.63s1501.14x
nvdec~20%~24%4.173s946.081.58x
x264enc~55%0%7.704s1298.024.03x
nvh264enc1 core 100% ~15%1.912s5230.12x

Example pipelines

Write to file

Note: mp4mux is used to mux stream in ISO MPEG-4 container format.

Display video

Common way to display video with autovideosink works around ~245 FPS on test machine:

Since, nvdec outputs buffers in raw format located on GPU (video/x-raw(memory:GLMemory)), so you can easily display it with glimagesink. This is great and fast OpenGL plugin to render video frames. With the following approach video displaying works with ~700 FPS.

And the last approach is to use nvdec for decoding, but autovideosink for video display. In such a case we need to transfer raw buffer from GPU to CPU memory with gldownload plugin:

Pipeline works with around 180 FPS (Due to additional memory conversion and allocation)

Note: for FPS measurements use fpsdisplaysink. For example:

Check out more pipelines at gstreamer-commands page

Common Issues

Version mismatch

Problem. Video Codec SDK Version not supported by Nvidia Driver.

Solution. Download proper SDK version to match your driver (go to: Install NVIDIA Video Codec SDK section) . Or change your driver version (if available in Software & Updates).

No errors in output

Problem. Plugins now working and no error messages printed in console.

Solution. Clear gstreamer cache to reload plugins. This should at least print error message

Conclusion

Using previous guide we learned how to:

Hope everything worked as expected 😉 In case of any troubles, suggestions, particular cases that haven’t been covered leave a comment.

References

Add a Comment

Your email address will not be published. Required fields are marked *