How to write Gstreamer plugin with Python?

     Next guide shows steps to write Gstreamer Plugin in Python for any Computer Vision, Image Processing task and use it in standard Gstreamer pipeline from command line. Additionally this post explains how to get/set own properties from implemented Gstreamer plugin.

     Inspired by guide “How to write Gstreamer elements in Python”, where the author shows how to write Audio Source and Filter Elements, I wanted to create simple example on how to write Gstreamer plugins for Computer Vision/Image Processing purpose. Despite mentioned disadvantages of Python implementation of Gstreamer elements it is still faster than any other Python approach in image processing (like OpenCV), and more flexible as you can easily reuse this element in other Gstreamer pipelines without changing any code.

     In previous post “How to create simple Blur Filter with Gstreamer in Python” there is already implemented plugin. That plugin is registered statically so we can’t use in from command line.

“plugin which is private to an application or library and contained within the application or library (as opposed to being shipped as a separate module file)”

     And now let make some changes to make plugin visible in global environment, so we can inject it in any Gstreamer Pipeline.

Requirements

Otherwise, look at this simple guide to install OpenCV from sources.

How it works?

     Use the following code to run it on your PC.

     Launch simple commands, to check how it works (plugin’s code).

Note: To see the difference I used basic video mixing pipeline to put two streams (with/without blur) in one window (Look at gstreamer cheat sheet, to learn more about classic pipelines)

gstreamer plugin in python template

     Additionally, launch simple streamer plugin that is just simplest template for any custom plugin:

gstreamer plugin in python template

gstreamer plugin in python template

     Notice that the properties “int-prop”, “float-prop”, “bool-prop”, “str-prop”, “pyobject-prop” successfully set to values specified by command line.

Guide

    #1. Gst-Python Installation

     Install gst-python with –with-libpython-dir flag. Highly recommended to use guide from “How to install Gstreamer from sources” (Gst-Python Section). Before going to next part check that test case is working and not error being printed

    #2. Implement Gst.Element as a Filter.

     A little Theory 😉

     Every Filter contains Sink (receive incoming buffers) and Src (send outcoming buffers).

gstreamer element src sink

    #3 Define Src/Sink Pads

     Sink and Src are Gst.Pad. Here is well defined meaning and functions of Pad. The essential algorithm to create Pad is to specify Pad Template first. Pad Template describes pad’s name, direction (sink, src), presense (always, sometimes, request), caps. To create Sink Pad Template that accepts buffers in RGB format use next code:
     From Pad Template we can create and add Sink Pad.

     As buffer flows from Sink to Src we need to define chain() function in order to process buffer and push it to Src Pad. Here is a good explanation of chain() function

     And, chain() function should be the implemented with the following template:

     Also we need to set event() function to pass events forward (from SINK to SRC) in pipeline: (documentation)

     And, event() function has next template (just push event to Src Pad):

     After previous steps SINK is ready to use, so add let’s add to Gst.Element:

     To create Src Pad Template that pushes buffers in RGB format use next code:

     Create Src Pad:


     In case of Src Pad we need to set event() function to pass events backwards (from SRC to SINK) in pipeline: (documentation)

     Template of event() function for SRC (just push event to Sink Pad):

     For Src Pad we need to set query() function (documentation). The purpose of Queries is to get information about element’s format, buffers duration or offset.

     Simple query() looks like the following:

     Now Src Pad is ready to be added to Gst.Element:

    #4 Implement chain() function

     The main purpose of plugin is to blur image. So we need to make Gst.Buffer writable (recap: How to make Gst.Buffer writable) and apply Gaussian Blur (using OpenCV) to it. Look at the following code:


     Notice that image processing is done only in here (In-place buffer change):

    #5 Register plugin

     In previous article “How to implement BlurFilter with Gstreamer in Python using OpenCV” plugin was registered statically, so it can’t be used from command line. In a such way plugin is accessible only during application run-time.  To use Gstreamer plugin from command line, just put next two lines in the end of the file:

     Additional Part 🙂

    #6 Properties

     Most of Gstreamer plugins have properties that are being initialized by user when specifying commands in terminal. As every Gst.Element is derived from GObject we can easily use it’s __gproperties__ metadata to define own properties. It’s well described in official documentation.
     So to create own property fill __gproperties__ dictionary (property-name: tuple) with your own properties and it’s description. Common template is the following

     So properties for gaussian_blur (sigmaX, sigmaY, kernel) plugin are defined by the following lines of code:

    To use own properties just override do_get_property() method and include implementation for custom properties handle.

     And override do_set_property() as well:


     Hope everything works as expected 😉 In case of troubles with running code leave comments or open an issue in Github.

 

Add a Comment

Your email address will not be published. Required fields are marked *