With gstreamer appsrc element it easy to push buffers (ex.: numpy arrays) into gstreamer pipeline. Developer can benefit from a variety of already implemented gstreamer plugins and display image in window, write frame to a video file or send buffers over TCP/HTTP.
Working with videos using OpenCV there were some problems with video display (can’t be launched in multiple threads) and video recording (high volume of recorded video files, or requirescustom library build with specific options).
With Gstreamer it is much easier as displaying/recording can be executed in multiple threads, is flexible due to variety of gstreamer plugins (tcp, http, window, video file, descriptor, ..). Of course there are some problems: installation (sometimes it is painful to install gstreamer), buffers duplication (when converting to Gst.Buffer).
Note: queue is added to store buffers in exact order without buffers drop in case of overload. Limit queue size to max-size-buffers to reduce memory consumption (as when overload the size of queue could be huge)
Write python script
Import classes to work with Gstreamer pipeline in Python.
from gstreamer import GstContext, GstPipeline
Initialize GstContext that hides all GLib.MainLoop routine to handle gstreamer events.
Initialize Gstreamer pipeline with a command defined previously (but omit gst-launch-1.0 keyword)
pipeline = GstPipeline(command)
Setup AppSrc element
In order to get appsrc from pipeline use next line of code
appsrc = pipeline.get_by_cls(GstApp.AppSrc) # get AppSrc
Now, let instruct gstreamer appsrc element that we will be dealing with timed buffers. Check out all the options in Gst.Format.
CAPS = "video/x-raw,format=RGB,width=640,height=480,framerate=30/1"
appsrc.set_caps(Gst.Caps.from_string(CAPS)) # set caps
Note: format (colorspace), width, height, fps are required for appsrc work properly
Additionally, setup appsrc to block accepting new coming buffers in case of internal queue overload, otherwise a lot of buffers are going to be dropped.
Attention, all previous setup should be done before pipeline started. For this purpose class GstPipeline has method on_pipeline_init that is going to be executed when Gst.Pipeline inialized but not started yet. Let override current method:
First, launch simple display of randomly generated numpy arrays in a window.
Display with framerate
Let’s, make it more difficult and check that video is playing with specified frame rate (30). For this purpose we’ll use fpsdisplaysink plugin that draws frame rate over a video or prints it to console.