How to install Gstreamer VAAPI plugins on Ubuntu?

10 min. read |

Video Acceleration API (VAAPI) is yet another way to use hardware acceleration to improve video processing performance (1.5-2 times faster than on CPU). The following guide helps to install Gstreamer VAAPI plugins on Ubuntu and provides solutions to most common issues.

Requirements

• Ubuntu
• Gstreamer
• Intel Graphics Card
• VAAPI

Scripts

• Dockerfile

Learn how to?

• install gstreamer-vaapi package and enable VAAPI hardware-accelerated video processing
• install-update Intel Graphics Drivers

Use gstreamer plugins:

• sources: videotestsrc, filesrc
• transforms: videoconvert, capsfilter, VAAPI (vaapih264enc, vaapih264dec, vaapipostproc)
• sinks: fakesink, ximagesink, vaapisink

Preface

VAAPI Gstreamer plugins is an alternative to NVIDIA’s ones (recall: “How to install NVIDIA Gstreamer Plugins on Ubuntu“) in order to enable hardware acceleration for video processing. For example, gst-video-analytics framework (check out: “Deep Learning Video Analytics Frameworks“) contains gstreamer plugins for video-analytics with OpenVino. To use framework the target platform must have Intel CPU and Intel Graphics Card. So, if you are restricted to Intel Hardware, then Video Acceleration API is a great choice. In the following guide we are going to provide steps in order to install drivers, requirements and gstreamer package for VA-API support .

Note: In gst-video-analytics framework Intel Media SDK works slower (see: Common Issues Section)

Guide

Environment Setup

• Ubuntu 18.04.4 LTS
• Gstreamer 1.14.5
• Intel Graphics Card 591b

Inspect

To check if VA-API plugins are already installed, use the following command:

gst-inspect-1.0 vaapi

Requirements

Check the official gstreamer requirements page for VA-API support.

Check Hardware

Use lshw utility to fetch important hardware information about Intel Graphics Card.

lshw -c video

Make sure that VGA compatible device with Intel Drivers present.

Note: install lshw with

apt-get install lshw

Also you can use lspci utility to check same device and drivers.

lspci | grep VGA

Note: install lspci with

apt install pciutils

In addition, you can list available devices in the following location:

ls /dev/dri

# card0  card1  renderD128  renderD129

Install Drivers

By default Intel Graphics driver is already installed on Ubuntu as xserver-xorg-video-intel. But, to update driver execute next commands:

add-apt-repository ppa:oibaf/graphics-drivers
apt update
apt dist-upgrade

# reboot

Now, install VA-API drivers.

apt install va-driver-all

In order to list available video drivers type next:

ls /usr/lib/x86_64-linux-gnu/dri | grep drv_video.so

# i965_drv_video.so
# nouveau_drv_video.so
# r600_drv_video.so
# radeonsi_drv_video.so

With vainfo (diagnostic tool for VA-API) check that everything is loaded correctly without any warning or mistakes.

vainfo

Note: install vainfo utility with

apt install vainfo

Note: Check that under line <supported profile entry points> there is a list of supported Video Accelerated operations.

Pay attention to requested driver ‘i965‘ and driver location /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so. To change this just edit the environmental variables

export LIBVA_DRIVER_NAME=i965
export LIBVA_DRIVERS_PATH=/usr/lib/x86_64-linux-gnu/dri

Or reset:

unset LIBVA_DRIVER_NAME
unset LIBVA_DRIVERS_PATH

Install gstreamer-vaapi

The safest way is to install Gstreamer’s VA-API plugin as official Ubuntu package:

apt install gstreamer1.0-vaapi

Otherwise, you can build it by yourself using the following commands.

git clone https://github.com/GStreamer/gstreamer-vaapi.git
cd gstreamer-vaapi

git checkout $(gst-launch-1.0 --version | \ 
grep version | tr -s ' ' '\n' | tail -1)

./autogen.sh --disable-gtk-doc --prefix=/usr

make 
sudo make install

Intel Graphics Card monitoring tool

To check Intel Graphics card load use the next command:

sudo intel_gpu_top

Note: Here is a brief explanation of intel-gpu-top output.

In addition, there is a prepared docker container that allows you to build everything as discussed previously.

VAAPI-accelerated pipelines

Hardware specifications

• GPU GeForce GTX 1050
• CPU Intel(R) Core(TM) i7-7700HQ @ 2.80GHz

Plugins performance

For performance evaluation we are going to use Jumanji: The Next Level Final Trailer (2019) (check out “Nvidia Gstreamer Plugins. Section: Nvidia-accelerated pipelines”). Video’s resolution and frames count:

ffprobe -v error -select_streams v:0 \
-show_entries stream=nb_frames,width,height \
-of default=noprint_wrappers=1 jumanji.mp4 

# width=1280
# height=720
# nb_frames=3948

Decoding (H264)

First, decode video with avdec_h264 (CPU-based plugin):

gst-launch-1.0 filesrc location=jumanji.mp4 ! qtdemux \
! h264parse ! avdec_h264 ! fakesink

# Execution ended after 0:00:09.159311292

• 

• 

Note: Intel Graphics Card remains in idle state (~0%) while CPU is almost loaded with work (70%).

Now, decode video with vaapih264dec:

gst-launch-1.0 filesrc location=jumanji.mp4 ! qtdemux ! h264parse ! \
 vaapih264dec  ! fakesink

# Execution ended after 0:00:05.598153461

• 

• 

Note: With VA-API decoding enabled, Intel Graphics GPU is busy, while CPU is less loaded than with avdec_h264.

Encoding (H264)

gst-launch-1.0 videotestsrc num-buffers=10000 ! x264enc ! fakesink 

# Execution ended after 0:00:33.894688475

gst-launch-1.0 videotestsrc num-buffers=10000 ! \
vaapih264enc !  fakesink sync=false

# Execution ended after 0:00:23.787508850

Color conversion (I420-RGBA)

Most Computer Vision/Deep Learning applications require RGB-colorspace and VA-API allows to do color conversion as well (unlike NVIDIA gstreamer plugins)

gst-launch-1.0 videotestsrc num-buffers=10000 ! video/x-raw,format=I420 ! \
videoconvert n-threads=0 ! video/x-raw,format=RGBA ! fakesink

# Execution ended after 0:00:09.026676223

gst-launch-1.0 videotestsrc num-buffers=10000 ! video/x-raw,format=I420 ! \
vaapipostproc format=rgba ! fakesink

# Execution ended after 0:00:09.276519198

Summary

Let’s put all the numbers to table and summarize everything.

Transform	CPU, %	GPU, %	Elapsed Time, ms	FPS	Perf.
avdec_h264	~73	–	9.159	~431	1.63x
vaapih264dec	~40	~98	5.598	~705	x
x264enc	~50	–	33.894	~295	1.42x
vaapih264enc	~20	69	23.788	~420	x
videoconvert	1 core ~80	–	9.027	~1100	x
vaapipostproc	1 core ~80	48	9.27	~1100	x

Note: Video Encoding/Decoding with VA-API can give up to 2 times performance improvement.With image color-space conversion (I420 -> RGBA) there is almost no difference in performance.

Example Pipelines

Write to file

gst-launch-1.0 videotestsrc num-buffers=10000 ! vaapih264enc ! h264parse \ 
! mp4mux ! filesink location=video.mp4

Display Video

gst-launch-1.0 filesrc location=jumanji.mp4 ! qtdemux ! h264parse !\
 vaapih264dec  ! vaapipostproc ! ximagesink sync=false

Common issues

Intel Media SDK support

Note: According to Gstreamer-VAAPI 1.16 and libva and Gstreamer VA-API Troubleshooting, – Intel Media SDK is going to be supported since Gstreamer 1.18 and not officially supported yet.

DRM_IOCTL_I915_GEM_APERTURE Failed

With the following exports by Intel Media SDK

export LIBVA_DRIVER_NAME=iHD
export LIBVA_DRIVERS_PATH=/opt/intel/mediasdk/lib64/

You can receive the next warning:

Note: Had received this type of message only using Intel Media SDK.

With the following warning most plugins work with less performance . For example, let’s do simple decoding:

GST_VAAPI_ALL_DRIVERS=1 gst-launch-1.0 filesrc location=jumanji.mp4 ! \
qtdemux ! vaapih264dec ! fakesink

# Execution ended after 0:01:10.107532193

gst-launch-1.0 filesrc location=jumanji.mp4 ! qtdemux ! h264parse ! \
 avdec_h264 ! fakesink

# Execution ended after 0:00:10.261924821

Decoding with VA-API is almost 10 times slower than common CPU-based plugin avdec_h264

In order to fix warning just import original drivers.

export LIBVA_DRIVER_NAME=i965
export LIBVA_DRIVERS_PATH=/usr/lib/x86_64-linux-gnu/dri

Additional. Some plugins (ex: vaapisink) causes Caught SIGSEGV with Intel Media SDK

For further reading checkout Gstreamer VA-API Troubleshooting

Conclusion

With Gstreamer VAAPI plugins you can enable hardware-accelerated video processing and perform encoding/decoding almost 2 times faster. Also with current approach you can parallel video processing to Intel Graphics Card and save CPU for other tasks. VA-API as part of Gstreamer is very useful for building applications based on gst-video-analytics framework

Hope everything worked as expected. In case of any issues, questions, suggestions contact me 😉

References

• Using hardware acceleration with Docker
• Gstreamer VA-API Troubleshooting
• Gstreamer-VA-API 1.16 and libva 2.6 in Debian
• Hardware-accelerated video decoding
• VAAPI-plugin