[gst-devel] sending extra data along with buffers

[Stefan Kost]

hi,

I've put a summary on the wiki:
http://gstreamer.freedesktop.org/wiki/BufferMetadata

please add your use-cases and ideas.

Stefan

Quoting Stefan Kost <ensonic at hora-obscura.de>:
> Hi,
>
> Jan Schmidt schrieb:
>> <quote who="Stefan Kost">
>>
>>> hi,
>>>
>>> I am looking for a way to send some private data along with a buffer.
>>> During data capture I get some metadata about capture settings and I would
>>> like to store/analyze them later along with the data stream. I tried to add
>>> a payload item to caps and put a GstBuffer there. But this seems to be
>>> painful and I am afraid that it will cause renegotiation as the content
>>> changes for each buffer. Has anyone been doing something similar?
>>>
>>> alternative (better approach) seems to be using a buffer subclass, right?
>>>
>>> Attached is my current test. Its a bit hackish and fails. One thing I was
>>> wondering is that fakesrc send buffer with caps=NULL. A bug?
>>>
>>
>>> From time to time, I've thought about adding something like this directly
>> to GstBuffer, for carrying information whose presence been negotiated in
>> the caps, but whose content should be allowed to change without
>> renegotiating.
>>
>> Examples:
>>   * Pan and scan (video subregion) position information
>>   * interlacing markers (top field, bottom field)
>>   * theora granulepos, instead of slapping it in the 'offset' field
>>   * MPEG Decoding timestamps, which we currently never send along.
>>
>> My thought was to add a GstStructure to GstBuffer which is allowed to be
>> NULL.
>>
>> Wim has expressed concern that it would cause performance issues, and it
>> would certainly require extending various bits of GStreamer to support
>> properly (buffer serialisation/deserialisation in GDP, for example). Also,
>> the semantics of how to handle the 'extradata' when transforming a buffer
>> need to be clearly defined, ie the extradata has to be clearly specified
>> by the caps attached to the buffer, and there needs to be a way for it to
>> be handled by elements that don't understand the extradata they've been
>> given. It can't just be a dumping ground in which to place arbitrary data,
>> or I don't think it can be made to work.
>>
>> J.
>>
> Maybe some GstTaglist features can help. I am thinking of the   
> merge-modes here.
> Atleast this is needed when mixing data.
>
> No idea how to express constraints that would render the data useless. E.g.
> muxing two streams removes the 'frame' context from video frames.   
> Thus the muxer
> would not copy/merge the extra-data to outgoing buffers.
>
> I am not so much concerned with performance. In most cases it would   
> not be used.
> Freeing buffers would need to release the taglst if its not NULL. Copying
> buffers need to copy it.
>
> As we have hierarchical structures now, we could probably require that the
> producer uses its element name as a root tag. This way the taglist   
> could carry
> multiple data-sets identified in a unique way. Elements that can use the data
> need to know the name of the element that created it.
>
> I'll think more about it - may be we can use our shiny new wiki to   
> draft it :)
>
> Stefan
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>