How to set Gstreamer Plugin rank? (Hardware priority)

5 min. read |

In the following guide we are going to take a look on how to set gstreamer plugin rank and prioritize some elements over the others when using playback tools (auto-plug).

Requirements

• Ubuntu 18
• Python 3.6
• Gstreamer with Python Bindings. Look at “How to install Gstreamer Python Bindings on Ubuntu 18”
• gstreamer-python

Code

• run_wrt_rank.py

Learn how to?

• set rank for gstreamer plugin in Python
• set plugin’s priority for playback tools
• use plugins: decodebin, rtspsrc, avdec_h264, NVIDIA DeepStream Gstreamer plugins, videoconvert, gtksink

Introduction

Gstreamer is flexible and plugin-based media streaming framework. A lot of plugins do the similar functions. For example, different hardware-accelerated video encoding and decoding plugins. Also Gstreamer has a great playback plugins that simplifies pipelines creation with auto-plugging and type finding tools. So, framework allows user to prioritize some plugins over the others.

For example, on my PC there are three different hardware accelerated video decoders: NVIDIA (Deepstream and Gstreamer plugins), VAAPI and Libav.

# plugin_name rank
nvv4l2decoder 267
vaapidecodebin 258
avdec_h264 256
avdec_h265 256
avdec_mpeg2video 256
avdec_mpeg4 256
avdec_mpegvideo 256
avdec_msmpeg4 256
avdec_rv10 256
flxdec 256
openexrdec 256
openjpegdec 256
theoradec 256
vaapih264dec 256
vaapih265dec 256
vaapimpeg2dec 256
vaapivc1dec 256
vaapivp8dec 256
vaapivp9dec 256

...

Recall how to install different hardware-accelerated video processing plugins:

• How to install NVIDIA Gstreamer plugins
• How to install VAAPI Gstreamer plugins

As you noticed the plugin with most rank is nvv4l2decoder (267). And here is a problem, whenever I want to simplify my pipeline with playback plugins (decodebin, …) those plugins are going to autoplug most ranked plugins (nvv4l2decoder in my case).

There are 3 scenarios to avoid such problem:

• disable hardware with unset of specific environment variables (ex: unset LIBVA_DRIVER_NAME to disable VAAPI)
• define pipelines without playback plugins (ex.: use “… ! qtdemux ! h264parse ! avdec_h264 ! …” instead of just ” … ! decodebin ! …”
• change rank of specific plugins

The choice is up to you and specification of problem. Here we are going to have a look at how to change rank of specific plugins.

Guide

At first, let’s list all available video decoders.

filt = Gst.ELEMENT_FACTORY_TYPE_DECODER  # only decoders
filt |= Gst.ELEMENT_FACTORY_TYPE_MEDIA_VIDEO  # only for video

Note: Element types defined with Gst.ELEMENT_FACTORY_TYPE_* and Media types defined with Gst.ELEMENT_FACTORY_TYPE_MEDIA_*. Have a look at Gst.Constants

To list all plugins by filter use list_get_elements command.

# returns list of Gst.ElementFactory
factories = Gst.ElementFactory.list_get_elements(filt, Gst.Rank.MARGINAL)

Note: MARGINAL rank is the lowest one (see Gst.Rank). So, in the following example we are listing all the plugins selected by filter with rank equal or more than MARGINAL rank.

Now, let’s sort by rank and print plugin’s names with ranks.

factories = sorted(factories, key=lambda f: f.get_rank(), reverse=True)
for f in factories:
    print(f.get_name(), f.get_rank())

Note: ElementFactory extends PluginFeature with get_name() and get_rank() methods

Listed plugins example:

# plugin_name rank
nvv4l2decoder 267
vaapidecodebin 258
avdec_h264 256
....

Now, let’s change avdec_h264 rank to incremented nvv4l2decoder‘s rank. So, avdec_h264 is going to be video decoder with most priority for playback plugins.

target_element = Gst.ElementFactory.find("avdec_h264")

# factories: sorted by rank in descending order
max_rank_element = factories[0]  # nvv4l2decoder

# Increment max element rank
target_element.set_rank(max_rank_element.get_rank() + 1)

Now, print plugin name and rank to check:

print("Rank of target plugin:", target_element.get_name(), "(", target_element.get_rank(), ")")

> avdec_h264 268 # > 267

Example

Let’s put everything together and run the following pipeline with python script: run_wrt_rank.py

First, have a look at pipeline running by default

gst-launch-1.0 rtspsrc location=rtsp://freja.hiof.no:1935/rtplive/definst/hessdalen03.stream ! decodebin ! fakesink -v

Note: free live RTSP source is taken from here.

As you can see at the following picture, decodebin plugs Nvidia plugins: nvv4l2decoder and nvvideoconvert.

Note: -v flag allows to inspect all plugins running in pipeline.

Note: Alternatively you can visualize your pipeline as a graph using the following guide

Finally, run pipeline using gstreamer-python.

pipeline_str = "rtspsrc location=rtsp://freja.hiof.no:1935/rtplive/definst/hessdalen03.stream ! decodebin ! fakeink"

with GstContext(), GstPipeline(pipeline_str) as p:
    while not p.is_done:
        time.sleep(1)

Put next lines at the end of script to print all elements that are present in pipeline:

elements = [el.get_factory().get_name() for el in p.pipeline.iterate_recurse()]
print("All elements: ", elements)
status = 'present' if target_element_name in set(elements) else "missing"
print(f"Target element ({target_element_name}) is {status}")

Run the script with the following command:

python examples/run_wrt_rank.py -d "avdec_h264" -p "rtspsrc location=rtsp://freja.hiof.no:1935/rtplive/definst/hessdalen03.stream ! decodebin ! videoconvert ! gtksink"

Note: In order to run script inside gstreamer-python repository specify PYTHONPATH:

PYTHONPATH=. python examples/run_wrt_rank.py ...

You should see something like that.

Here are all plugins that are being used in pipeline.

All elements:  'gtksink', 'videoconvert', 'decodebin', 'avdec_h264', 'capsfilter', 'h264parse', 'rtph264depay', 'typefind', 'rtspsrc', 'udpsink', 'fakesrc', 'udpsink', 'udpsrc', 'udpsrc', 'udpsink', 'fakesrc', 'udpsink', 'udpsrc', 'udpsrc', 'rtpbin', 'rtpjitterbuffer', 'rtpptdemux', 'rtpjitterbuffer', 'rtpptdemux', 'rtpstorage', 'funnel', 'funnel', 'rtpssrcdemux', 'rtpsession', 'rtpstorage', 'funnel', 'funnel', 'rtpssrcdemux', 'rtpsession'

And finally our target plugin is present in pipeline (and there is no Nvidia plugins).

Target element (avdec_h264) is present

Conclusion

Now, you can simplify your pipelines with auto tools like (decodebin, …) and use plugins with the highest priorities.

Hope everything works as expected 😉 In case of errors, troubles, suggestions – leave comments or contact me via social pages.

References

• How to generate a Gstreamer pipeline diagram (graph)
• Getting pipeline graphs
• Typefind plugin’s rank