How to install Gstreamer VAAPI plugins on Ubuntu?
10 min. read |
Video Acceleration API (VAAPI) is yet another way to use hardware acceleration to improve video processing performance (1.5-2 times faster than on CPU). The following guide helps to install Gstreamer VAAPI plugins on Ubuntu and provides solutions to most common issues.
- Intel Graphics Card
Learn how to?
- install gstreamer-vaapi package and enable VAAPI hardware-accelerated video processing
- install-update Intel Graphics Drivers
Use gstreamer plugins:
- sources: videotestsrc, filesrc
- transforms: videoconvert, capsfilter, VAAPI (vaapih264enc, vaapih264dec, vaapipostproc)
- sinks: fakesink, ximagesink, vaapisink
VAAPI Gstreamer plugins is an alternative to NVIDIA’s ones (recall: “How to install NVIDIA Gstreamer Plugins on Ubuntu“) in order to enable hardware acceleration for video processing. For example, gst-video-analytics framework (check out: “Deep Learning Video Analytics Frameworks“) contains gstreamer plugins for video-analytics with OpenVino. To use framework the target platform must have Intel CPU and Intel Graphics Card. So, if you are restricted to Intel Hardware, then Video Acceleration API is a great choice. In the following guide we are going to provide steps in order to install drivers, requirements and gstreamer package for VA-API support .
Note: In gst-video-analytics framework Intel Media SDK works slower (see: Common Issues Section)
- Ubuntu 18.04.4 LTS
- Gstreamer 1.14.5
- Intel Graphics Card 591b
To check if VA-API plugins are already installed, use the following command:
Check the official gstreamer requirements page for VA-API support.
Use lshw utility to fetch important hardware information about Intel Graphics Card.
lshw -c video
Make sure that VGA compatible device with Intel Drivers present.
Note: install lshw with
apt-get install lshw
Also you can use lspci utility to check same device and drivers.
lspci | grep VGA
Note: install lspci with
apt install pciutils
In addition, you can list available devices in the following location:
ls /dev/dri # card0 card1 renderD128 renderD129
By default Intel Graphics driver is already installed on Ubuntu as xserver-xorg-video-intel. But, to update driver execute next commands:
add-apt-repository ppa:oibaf/graphics-drivers apt update apt dist-upgrade # reboot
Now, install VA-API drivers.
apt install va-driver-all
In order to list available video drivers type next:
ls /usr/lib/x86_64-linux-gnu/dri | grep drv_video.so # i965_drv_video.so # nouveau_drv_video.so # r600_drv_video.so # radeonsi_drv_video.so
With vainfo (diagnostic tool for VA-API) check that everything is loaded correctly without any warning or mistakes.
Note: install vainfo utility with
apt install vainfo
Note: Check that under line <supported profile entry points> there is a list of supported Video Accelerated operations.
Pay attention to requested driver ‘i965‘ and driver location /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so. To change this just edit the environmental variables
export LIBVA_DRIVER_NAME=i965 export LIBVA_DRIVERS_PATH=/usr/lib/x86_64-linux-gnu/dri
unset LIBVA_DRIVER_NAME unset LIBVA_DRIVERS_PATH
The safest way is to install Gstreamer’s VA-API plugin as official Ubuntu package:
apt install gstreamer1.0-vaapi
Otherwise, you can build it by yourself using the following commands.
git clone https://github.com/GStreamer/gstreamer-vaapi.git cd gstreamer-vaapi git checkout $(gst-launch-1.0 --version | \ grep version | tr -s ' ' '\n' | tail -1) ./autogen.sh --disable-gtk-doc --prefix=/usr make sudo make install
Intel Graphics Card monitoring tool
To check Intel Graphics card load use the next command:
Note: Here is a brief explanation of intel-gpu-top output.
In addition, there is a prepared docker container that allows you to build everything as discussed previously.
- GPU GeForce GTX 1050
- CPU Intel(R) Core(TM) i7-7700HQ @ 2.80GHz
For performance evaluation we are going to use Jumanji: The Next Level Final Trailer (2019) (check out “Nvidia Gstreamer Plugins. Section: Nvidia-accelerated pipelines”). Video’s resolution and frames count:
ffprobe -v error -select_streams v:0 \ -show_entries stream=nb_frames,width,height \ -of default=noprint_wrappers=1 jumanji.mp4 # width=1280 # height=720 # nb_frames=3948
First, decode video with avdec_h264 (CPU-based plugin):
gst-launch-1.0 filesrc location=jumanji.mp4 ! qtdemux \ ! h264parse ! avdec_h264 ! fakesink # Execution ended after 0:00:09.159311292
Note: Intel Graphics Card remains in idle state (~0%) while CPU is almost loaded with work (70%).
Now, decode video with vaapih264dec:
gst-launch-1.0 filesrc location=jumanji.mp4 ! qtdemux ! h264parse ! \ vaapih264dec ! fakesink # Execution ended after 0:00:05.598153461
Note: With VA-API decoding enabled, Intel Graphics GPU is busy, while CPU is less loaded than with avdec_h264.
gst-launch-1.0 videotestsrc num-buffers=10000 ! x264enc ! fakesink # Execution ended after 0:00:33.894688475
gst-launch-1.0 videotestsrc num-buffers=10000 ! \ vaapih264enc ! fakesink sync=false # Execution ended after 0:00:23.787508850
Color conversion (I420-RGBA)
Most Computer Vision/Deep Learning applications require RGB-colorspace and VA-API allows to do color conversion as well (unlike NVIDIA gstreamer plugins)
gst-launch-1.0 videotestsrc num-buffers=10000 ! video/x-raw,format=I420 ! \ videoconvert n-threads=0 ! video/x-raw,format=RGBA ! fakesink # Execution ended after 0:00:09.026676223
gst-launch-1.0 videotestsrc num-buffers=10000 ! video/x-raw,format=I420 ! \ vaapipostproc format=rgba ! fakesink # Execution ended after 0:00:09.276519198
Let’s put all the numbers to table and summarize everything.
Note: Video Encoding/Decoding with VA-API can give up to 2 times performance improvement.With image color-space conversion (I420 -> RGBA) there is almost no difference in performance.
Write to file
gst-launch-1.0 videotestsrc num-buffers=10000 ! vaapih264enc ! h264parse \ ! mp4mux ! filesink location=video.mp4
gst-launch-1.0 filesrc location=jumanji.mp4 ! qtdemux ! h264parse !\ vaapih264dec ! vaapipostproc ! ximagesink sync=false
Intel Media SDK support
Note: According to Gstreamer-VAAPI 1.16 and libva and Gstreamer VA-API Troubleshooting, – Intel Media SDK is going to be supported since Gstreamer 1.18 and not officially supported yet.
With the following exports by Intel Media SDK
export LIBVA_DRIVER_NAME=iHD export LIBVA_DRIVERS_PATH=/opt/intel/mediasdk/lib64/
You can receive the next warning:
Note: Had received this type of message only using Intel Media SDK.
With the following warning most plugins work with less performance . For example, let’s do simple decoding:
GST_VAAPI_ALL_DRIVERS=1 gst-launch-1.0 filesrc location=jumanji.mp4 ! \ qtdemux ! vaapih264dec ! fakesink # Execution ended after 0:01:10.107532193
gst-launch-1.0 filesrc location=jumanji.mp4 ! qtdemux ! h264parse ! \ avdec_h264 ! fakesink # Execution ended after 0:00:10.261924821
Decoding with VA-API is almost 10 times slower than common CPU-based plugin avdec_h264
In order to fix warning just import original drivers.
export LIBVA_DRIVER_NAME=i965 export LIBVA_DRIVERS_PATH=/usr/lib/x86_64-linux-gnu/dri
Additional. Some plugins (ex: vaapisink) causes Caught SIGSEGV with Intel Media SDK
For further reading checkout Gstreamer VA-API Troubleshooting
With Gstreamer VAAPI plugins you can enable hardware-accelerated video processing and perform encoding/decoding almost 2 times faster. Also with current approach you can parallel video processing to Intel Graphics Card and save CPU for other tasks. VA-API as part of Gstreamer is very useful for building applications based on gst-video-analytics framework
Hope everything worked as expected. In case of any issues, questions, suggestions contact me 😉
Thank you for a very clear article, with the great tips of checking every step’s successful installation. I have 2 questions:
1. Do I need to use gstreamer-vaapi for intel hw accelaration using media sdk.
In this here, they dont say anything for gstreamer-vaapi:
it says to use mfx-prefixed libraries, but then i get this error:
gst-launch-1.0 –gst-debug=3 filesrc location=/home/radiant/videos/file_example_MP4_1920_18MG.mp4 ! decodebin name=demux demux. ! queue ! audioresample ! audioconvert ! avenc_aac ! mux. mpegtsmux alignment=7 name=mux ! filesink location=/home/radiant/videos/out_sample_1280x720.ts demux. ! queue ! videoconvert ! videoscale ! mfxh264enc ! video/x-h264,stream-format=byte-stream,profile=high ! mux.
Setting pipeline to PAUSED …
DRM_IOCTL_I915_GEM_APERTURE failed: Bad file descriptor
Assuming 131072kB available aperture size.
May lead to reduced performance or incorrect rendering.
get chip id failed: -1 
param: 4, val: 0
0:00:00.031937038 190916 0x5637a723acc0 WARN videoencoder gstvideoencoder.c:1774:gst_video_encoder_change_state: error: Failed to open encoder
ERROR: from element /GstPipeline:pipeline0/GstMfxEncH264:mfxench264-0: Could not initialize supporting library.
2. Today, I followed the steps you have outlined and validated them. Also Installed gstreamer1.0-vaapi but nothing shows when i do a gst-inspect-1.0|grep vaapi
gstreamer1.0-vaapi had got installed ok
sudo apt install gstreamer1.0-vaapi
Reading package lists… Done
Building dependency tree
Reading state information… Done
gstreamer1.0-vaapi is already the newest version (1.16.2-2).
my LIBVA_DRIVER was earlier IHD, but i changed it to i965.