[Bug 774049] New: New features/functionality in the Aggregator and VideoAggregator

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=774049

            Bug ID: 774049
           Summary: New features/functionality in the Aggregator and
                    VideoAggregator
    Classification: Platform
           Product: GStreamer
           Version: 1.x
                OS: Linux
            Status: NEW
          Severity: enhancement
          Priority: Normal
         Component: gst-plugins-bad
          Assignee: gstreamer-bugs at lists.freedesktop.org
          Reporter: msb at qtec.com
        QA Contact: gstreamer-bugs at lists.freedesktop.org
     GNOME version: ---

In the recent Gstreamer conference (2016) I have presented a talk about using
Gstreamer pipelines for image processing in industrial applications.
In this talk I mentioned that I had the need of muxing some metadata generated
by processing (and thus modifying) image frames into the original (unmodified)
images.

For this purpose I have created my own muxing element (using the Compositor as
inspiration) which uses the VideoAggregator as the base class.
However the VideoAggreagtor and it's base class the Aggregator didn't quite
cover my needs so I ended up modifying my own local copies of these 2 classes.

During the conference there was interest expressed in hearing which where my
different use cases because maybe they could be added upstream.

Since I want to mux what is actually the same original frame (just divided and
processed differently by 2 separate tee branches) I want to do timestamp
matching of the buffers (I don't care about maintaining the fps and don't want
to duplicate frames):

-Therefore I want to produce output frames only when I have received input
frames in all pads (in contrast to the Aggregator producing output as soon as 1
pad has data, in case of a live source which is what I have).
My easy solution for this was adding a new property to the Aggregator that
forces it to work as with non-live sources (so that it requires data in all
pads).

-Then I added to VideoAggregator a property that defines if it should work
"normally" (producing frames in a deadline, fps) or if it should check the
buffer timestamps and only produce frames when they match. I have created a
function like the fill_queues() function which does the timestamp checking.

-In case I'm only producing frames based on matched timestamps I have problems
with QOS because the segment times might not match anymore (since frames are
not necessarily produced regularly anymore). To solve this I have created a
function that adjusts the segment times based on the produced frames.

-Lastly, in my "Compositor like" Muxer element I needed the option to drop
frames in some cases so I have added a new GstFlowReturn to VideoAggregator
(GST_FLOW_VIDEO_AGGREGATOR_DROP_FRAME == GST_FLOW_CUSTOM_SUCCESS) just like it
exists for the BaseTransform

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.