How to track buffer caps in gstreamer 1.x ?

[Thiago Santos]

On 01/29/2015 12:52 PM, Ben Bridgwater wrote:
> Thanks, Thiago.
>
> Yes, in my application (because of the out-of-order buffer processing) 
> I do need to explicitly associate buffers with caps rather than just 
> rely on the caps event and buffer ordering.
What is your application exactly? Are you using appsink, fakesink or a 
custom sink? Or is this another type of element?
>
> I assume gstreamer pipeline elements can't actually pass GstSamples 
> rather than GstBuffers - this is just a convenience class for a sink 
> to associate buffers/caps/segments rather than a type that would be 
> passed from element to element?
Yes, only buffers are passed through pads.
>
> It seems another option for me might be to use GstBuffer metadata to 
> add the tagging/caps info I need. Would I be right in thinking that an 
> element could receive a GstBuffer, add some metadata, then pass the 
> GstBuffer with added metadata down the pipleline? Would an existing 
> element/plugin care if it received a buffer with added metadata that 
> it wasn't expecting? Presumably adding metadata involves an additional 
> malloc?
>
> Is there any way to add any metadata (maybe just an integer tag) to a 
> GstBuffer without doing an additional alloc? Any GstBuffer members 
> than it would be harmless to repurpose as a tag, perhaps?
There is GstMeta that is a flexible metadata system that can be 
associated with buffers. You need to make sure that other elements in 
the pipeline preserve the meta when passing buffers around (specially 
the ones that create new buffers for outputing, like decoders). There 
are also some flags in GstBuffer itself that might be useful for you.

In any case, associating a Caps to a Buffer is a bad idea as it is 
already implicitly associated by the previous caps event.

Perhaps you could provide more details on your use case so we can 
provide better guidance.
>
> Thanks,
> Ben
>
> On Thu, Jan 29, 2015 at 10:05 AM, Thiago Santos 
> <thiagoss at osg.samsung.com <mailto:thiagoss at osg.samsung.com>> wrote:
>
>     On 01/29/2015 11:53 AM, Ben Bridgwater wrote:
>>     Hi,
>>     I'm new to gstreamer development - converting an existing
>>     application (interactive spectrogram) from using my own pipeline
>>     to use gstreamer instead.
>>
>>     I gather that in gstreamer 0.1 the caps of a pad sourcing a
>>     buffer were also recorded in the buffer itself, but since 1.0
>>     that is no longer the case, and it's up to elements to tracks
>>     caps via caps events instead...
>>
>>     My question is what are the options and best practices with
>>     gstreamer 1.x for tracking buffer caps in the case where caps are
>>     changing rapidly and buffers may be processed out of order? In my
>>     case this requirement comes from a downstream display buffer
>>     where paint events may request arbitrary parts of the display to
>>     be updated - corresponding to arbitrary sequences of buffers
>>     needing to be processed, and the need to know what the caps (data
>>     format) are for each buffer.
>
>     The GstCaps are not stored on buffers anymore, they are only
>     present on GstPads. Instead of calling gst_pad_set_caps or pushing
>     a buffer with a caps set, now elements should push a caps event
>     before the buffers and all buffers following a caps event should
>     have the caps of that event. You can use GstPad's API to get the
>     current negotiated caps on a pad and all buffers flowing on that
>     pad should have that caps until a new event arrives. Check
>     http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstPad.html#gst-pad-get-current-caps
>
>     So, to keep track of buffers and caps, you need to look for the
>     caps event. For example, suppose you receive the following events
>     and buffers:
>
>     CAPS_1 BUF_A BUF_B BUF_C CAPS_2 BUF_D BUF_E
>
>     Buffers A B and C will be on format defined by CAPS_1, while
>     buffers D and E will be on format CAPS_2.
>
>     I hope it makes it clear. If for some reason you need to store
>     buffer + caps together you can use the GstSample :
>     https://developer.gnome.org/gstreamer/stable/gstreamer-GstSample.html
>
>>
>>     Thanks for any pointers and advice!
>>
>>     Ben
>>
>>
>>
>>     _______________________________________________
>>     gstreamer-devel mailing list
>>     gstreamer-devel at lists.freedesktop.org  <mailto:gstreamer-devel at lists.freedesktop.org>
>>     http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
>     -- 
>     Thiago Sousa Santos
>     Senior Multimedia Engineer, Open Source Group
>     Samsung Research America - Silicon Valley
>
>
>     _______________________________________________
>     gstreamer-devel mailing list
>     gstreamer-devel at lists.freedesktop.org
>     <mailto:gstreamer-devel at lists.freedesktop.org>
>     http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
>
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


-- 
Thiago Sousa Santos
Senior Multimedia Engineer, Open Source Group
Samsung Research America - Silicon Valley

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20150129/98743385/attachment.html>