[gst-devel] neutral buffer handling in sparse streams

[Stefan Kost]

hi,

multitrack applications (those which compose media snippet to make a song or
video) usually have huge gaps in the tracks. That is e.g. between notes in a
synthesizer plugin or between video clips. Having post-processing elements
always process silence or alpha frames is wasting cpu cycles and thus limits the
tracks that can be used.

I found GST_BUFFER_FLAG_GAP and used that and it works quite nicely. Now this is
defined as "the buffer has been created to fill a gap in the stream" and I look
for something with the semantics "the buffer has neutral content". "Neutral
content" does not mean it can just be skipped, it just means that the element
can switch to a faster code path for this buffer. Some examples:

Volume:
  just pass through
Mixer:
  skip the buffer
Echo:
  drain the ring-buffer, but shortcut in the feedback loop

I don't think that using segments will work. Of couse it sounds tempting, but it
would need a lot of work on elements like adder and tee (and this needs a
concept of how that should be done there). It would also mean that elements like
echo need to transform the segment-boundaries, where as now they just don't care
about segments that much.

So what the way to go?

1) use GST_BUFFER_FLAG_GAP and define it as such
btw. its right now only used in gst-plugins-base/gst/audiorate/gstaudiorate.c to
mark a silent filler-buffer

2) add GST_BUFFER_FLAG_NEUTRAL

3) segments

4) anything else?


Stefan