[gst-devel] empty buffers & GST_BUFFER_FLAG_GAP

[Stefan Kost]

Hello Wim,

Wim Taymans schrieb:
> On Wed, 2006-01-18 at 12:47 +0100, Stefan Kost wrote:
>>hi,
> Hi Stefan,
> 
>>imagine a source that from time to time produces empty buffers (silence
>>or bank images). If the pipeline has many elements next, it would be
>>cool to optimize the obsolete data procession in this case.
>>
>>Examples for such sources are sound-generator (simsyn in gst-buzztard)
>>and also a source in voip applications that uses noise-gating (to save
>>bandwith).
> 
> We're writing some docs on how to handle sparse data streams such as
> subtitles and other usecases similar to this one. What you need is a way
> to notify elements that stream time progresses without sending any
> buffers. This is only needed for elements that mix/mux so that they know
> they should not wait for data for some period.
> 
> In 0.8 this was done with a filler event. Many other alternatives are
> possible such as sending empty buffers, marking buffers with a flag,
> sending some sort of event, updating the segment start value, ...
> 
>>What I like to do is having a GstBufferFlag that signals that this
>>buffer contains blank data. Subsequent elements can use that to spare
>>processing.
> 
> If you do this, are you going to put data in the buffer or have it be a
> size of 0?
> If the buffer has a size of 0, you could just simply assume that this is
> your "blank" buffer for the indicated TIME/DURATION/OFFSET/OFFSET_END
> without needing a flag. Plugins need to be updated to not process these
> blank buffers but pass them downstream (after possibly updating time and
> offset values).

My plan was to send a buffer with cleared memory. This way elements
*can* take the advantage of skipping to process it and dumb elements can
 continue to process them.

>>Is that what GST_BUFFER_FLAG_GAP is for? If not, may I also add
>>GST_BUFFER_FLAG_BLANK. As part of the proposal I would update GstVolume,
>>GstLevel, GstAdder and GstAudioconvert to make use of that.
> 
> The GAP flag was added to notify elements that this buffer was generated
> by some element to indicate that this buffer was inserted into the
> stream to fill a gap. Audiorate for example sets this flag when it needs
> to insert buffers to make a perfect stream. Not sure why we would ever
> need this, it was a feature request from Thomas, I'm sure he has a use
> case somewhere.

I grepped the whole source tree and found no reference to those flags.
So its probaly unused. If nobody knows otheriwse I will mark it in the
source with a (FIXME).

> There is a related FLAG_PREROLL that is actually made to mark buffers
> that should not be processed or rendered. Its name is probably wrong,
> though, but could be used to mark useless buffers.
> 
>>What do you think about it?
> 
> We're leaning towards using the segment start value to make stream time
> progress in the case of sparse data streams. The reason is that we
> effectively already use this mechanism to perform segment seeks and
> looping. Also most elements and filters do not need to bother at all
> with checking buffer flags and can just use the existing segment event
> code without any change.
> 
> The only addition needs to be done in mixing/muxing elements (adder) so
> that they know that they should not expect data on a pad with timestamps
> before the updated segment start.
> 
I understand that from the a processing elements pov. They won't notice
that they were not given all buffers, Wouldn't that cause a problem for
elements like an audio delay or the level element. An audio delay would
not be given a chance to flush its ring-buffer and the level element can
not slowly release the levels.

Using zero'ed buffer this would work. Its up to the element to make use
of the buffer-flag. Ideally those zero'ed buffers would be static, so
that for a longer perio of silence, we don't alloc empty memory again
and again.

> Wim
>>Stefan
>>
Stefan