How to create simple BlurFilter with Gstreamer in Python using OpenCV

     Now, let make simple Blur Video Filter using Gstreamer in Python. There is an existing official implementation of Gaussian Blur Filter. But, the main goal of the following post is to build practical background of Gstreamer.

Requirements

  • Python3
  • Gstreamer. I hope you have installed Gstreamer. If not – look at this post “How to install Gstreamer on Ubuntu”.
  • Numpy
  • OpenCV. This library is used for image blur, so if you want you can do it with Scipy, PIL. Otherwise, look at this simple guide to install OpenCV.

Note: I often use tutorials from LearnOpenCV, so if you want to be great Computer Vision Engineer become a frequent visitor of LearnOpenCV and PyImageSearch. The authors have perfect, understandable and clear material style in their posts. So I will often refer to practical guides from those websites 🙂

  • Clean your workspace 🙂

     Now, take a look at this Github repository. This post is based on that code.

     To run code, type next command in terminal(video is here):

     You should see something like this:

How to create Blur Filter with OpenCV and Gstreamer

     Now run with blur enabled:

How to create BlurFilter with OpenCV and Gstreamer in Python

     I hope everything working, and now you can see that for the second launch there is blur filter in action.

     Let’s go through main steps for implementing this simple plugin.

#1. Build Pipeline

     Gstreamer Pipeline is simple container of ordered elements through which data flows in specific direction, from Src to Sink

     For this task we will use next plugins:

  • filesrc (Src element. Reads data from file. Just specify location property)

  • decodebin (In order to read video, we need to parse specific video format. Decodebin handles this parsing pipeline under the hood)
  • videoconvert (Each element in Pipeline can have different formats. For example, some of elements process RGB buffers, some BGR, others YUV. To make this elements able to link each other we use videoconvert plugin)
  • gtksink (Sink element that allow to display video in window)

     Now, you should be able to launch next command in terminal. For this purpose use gst-launch-1.0

     In addition, you can go through “How to launch gstreamer pipeline from Python” to launch previous command from code and to get yourself ready to next steps.

#2. Implement Blur Filter Plugin

  • Define GstBlurFilter class:

     Pay attention to parent class GstBase.BaseTransform. It’s a base class for filter to process data in pipeline.

Note: Use next information to fill __gstmetadata__ in right way:

     First, let get size of inbuffer where pixel data collected. Look at post “How to get width, height of Gst.Buffer” to understand how next call works.

     Now, for image processing let convert Gst.Buffer to numpy array. For this purpose we need to access raw data from Gst.Buffer and create np.ndarray with already known size (width, height, channels). Look at “How to make Gst.Buffer writable” to understand map_gst_buffer function.

Note: Pay attention that CHANNELS=3, because plugin’s Src’s Caps corresponding to RGB format(“video/x-raw,format=RGB”)

     Now frame is ready for any manipulations. Let apply blur filter:

     To pass blurred frame to next gstreamer plugins we need to modify outbuffer with new data. For this purpose just copy blurred frame into mapped outbuffer:

Hack: Before writing into out buffer we are setting outbuffer’s mini_object.refcount to 1. It’s because the buffer is only writable when it’s refcount exactly 1, so only current plugin owns this buffer. We force buffer to have refcount exactly 1 and return back the refcount after all modifications. (The analog of this technique in C is gst_buffer_ref, gst_buffer_unref

 The buffer refcount determines the writability of the buffer, a buffer is only writable when the refcount is exactly 1, i.e. when the caller has the only reference to the buffer. (Source)

#4. Register plugin

     Now we need to register plugin in proper way, so we can use an element through calling Gst.make_element() or Gst.parse launch(). Here you can read more on how to use Gst.make_element and Gst.parse launch.

     Next function registers plugin statically for usage only in current application, so it would not be able to use plugin from terminal.


     To register GstBlurFilter as static plugin just call:

     Now your plugin is ready to use. Hope everything working as expected 😉

One Comment

Add a Comment

Your email address will not be published. Required fields are marked *