Video analytics applications (ex.: smart cities, retail, industries, etc.) consist of two main parts: Video Streaming and Computer Vision/ Deep Learning Frameworks. In here we’ll go through available frameworks that allow developers to focus on providing analytics part and hide nuances of video streaming.
General architecture of Video Analytics applications looks like the following.
Most Product Development Process from Client perspective could be reduced to follows (Video Analytics Case):
Reduce Development Costs/Time
existing solutions reuse
balance between Software Engineering and Data Science common skill set
Reduce Product Cost
process multiple video processing feeds; fast and accurate models usage
efficient hardware usage
shared memory resources
up to 99% hardware capabilities usage
reduced data storage and transmission
Reduce Product Scaling Costs/Time
generability: single solution multiple use cases
All listed frameworks helps to build software faster, in more efficient way.
I started to prototype Video Analytics applications from OpenCV. When we exceeded the limits of it due to new project requirements (resolution/fps setup, video record, custom operations, performance improvement) we switched to Gstreamer (advice by another expert).
I know C/C++ well, but started to dive deep into Gstreamer with Python (easier dependencies setup and development itself). With Python I was mostly focused on how framework works. So when prototyping we failed/succeeded faster.
Exploring Gstreamer is challenging but rewarding process. Luck of resources, community is a huge problem. The main pain was to setup everything and make Python friends with Gstreamer.
Diving into Gstreamer helped me to learn it’s architecture, code development approaches, basics of video processing. It was and still is an entertaining process 😉 . I think that Gstreamer has one of the best architecture (interfaces, abstraction) which gives developers great flexibility and extensibility (sometimes code might be dirty, not intuitive, but nothing is perfect …) 😉
Now, I’m glad when I see how other companies use Gstreamer for Video Analytics applications.
FFmpeg I use often as a command line tool (commands are shorter, sometimes more clear).
OpenCV works for me when there is a need to deliver prototype in a short time and there are no restrictions on Hardware performance. OpenCV supports Gstreamer as well but requires additional library build with enabled additional properties (with pip-package it is so much easier).
I constantly look for new repositories, frameworks which simplifying development of Video Analytics Applications.