[gst-devel] sending extra data along with buffers

Stefan Kost ensonic at hora-obscura.de
Tue Jan 15 21:56:22 CET 2008


Hi,

Jan Schmidt schrieb:
> <quote who="Stefan Kost">
> 
>> hi,
>>
>> I am looking for a way to send some private data along with a buffer. 
>> During data capture I get some metadata about capture settings and I would 
>> like to store/analyze them later along with the data stream. I tried to add 
>> a payload item to caps and put a GstBuffer there. But this seems to be 
>> painful and I am afraid that it will cause renegotiation as the content 
>> changes for each buffer. Has anyone been doing something similar?
>>
>> alternative (better approach) seems to be using a buffer subclass, right?
>>
>> Attached is my current test. Its a bit hackish and fails. One thing I was 
>> wondering is that fakesrc send buffer with caps=NULL. A bug?
>>
> 
>>From time to time, I've thought about adding something like this directly
> to GstBuffer, for carrying information whose presence been negotiated in
> the caps, but whose content should be allowed to change without
> renegotiating.
> 
> Examples:
>   * Pan and scan (video subregion) position information
>   * interlacing markers (top field, bottom field)
>   * theora granulepos, instead of slapping it in the 'offset' field
>   * MPEG Decoding timestamps, which we currently never send along.
> 
> My thought was to add a GstStructure to GstBuffer which is allowed to be
> NULL.
> 
> Wim has expressed concern that it would cause performance issues, and it
> would certainly require extending various bits of GStreamer to support
> properly (buffer serialisation/deserialisation in GDP, for example). Also,
> the semantics of how to handle the 'extradata' when transforming a buffer
> need to be clearly defined, ie the extradata has to be clearly specified
> by the caps attached to the buffer, and there needs to be a way for it to
> be handled by elements that don't understand the extradata they've been
> given. It can't just be a dumping ground in which to place arbitrary data,
> or I don't think it can be made to work.
> 
> J.
> 
Maybe some GstTaglist features can help. I am thinking of the merge-modes here.
Atleast this is needed when mixing data.

No idea how to express constraints that would render the data useless. E.g.
muxing two streams removes the 'frame' context from video frames. Thus the muxer
would not copy/merge the extra-data to outgoing buffers.

I am not so much concerned with performance. In most cases it would not be used.
Freeing buffers would need to release the taglst if its not NULL. Copying
buffers need to copy it.

As we have hierarchical structures now, we could probably require that the
producer uses its element name as a root tag. This way the taglist could carry
multiple data-sets identified in a unique way. Elements that can use the data
need to know the name of the element that created it.

I'll think more about it - may be we can use our shiny new wiki to draft it :)

Stefan





More information about the gstreamer-devel mailing list