Passing buffers upstream

[Tim-Philipp Müller]

On Sun, 2013-07-28 at 11:46 +0200, Peter Maersk-Moller wrote:

Hi,

have you looked at GstBufferPool and the ALLOCATION query yet?

Also
http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-bufferpool.txt
http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-buffer.txt

Cheers
 -Tim


> I wonder if the GStreamer API offers the possibility of a plugin to
> pass an allocated buffer upstream? An would it be possible for a
> plugin to allocate/construct the buffer from its own memory allocation
> scheme? Technically it would be so difficult, if the API has the right
> hooks. More difficult would it probably be, if a module would like
> pass one of its own allocated buffer downstream, but needs to be told,
> when the buffer is free again.
> 
> 
> Passing upstream is very useful if the plugin module shmsink could
> pass a buffer upstream that points to an area in its shared memory.
> Likewise the shmsrc, could pass a buffer downstream pointing to its
> shared memory. I understand that the two cases are technical
> different, especially the downstream is difficult, as the consumer of
> the buffer must signal upstream, that the buffer has been used and are
> free for reuse by the originator of the buffer.
> 
> 
> Why is this useful? Imagine you have a video mixer (like Snowmix) that
> needs its input video feeds from shmsink and imagine there are mutiple
> video feeds like 10 feeds and imagine these feeds runs as 1080p at
> 30fps and imagine the format is RGBA for optimal quality, then that
> amounts to around 245MB per second per feed. That is really a lot of
> data. Being able to limit the amount of unnecessary data transfer
> would really help. Imagine a pipeline like below
> 
> 
>   gst-launch  some_video_src ! ffdec_h264 ! ffmpegcolorspace ! shmsink
> 
> 
> Parameters are left out in this example and the example uses gst
> version 0.10 modules. Version 1.x is slightly different. In the
> example, the system would save a memory transfer if the shmsink module
> could pass a buffer to ffmpegcolorspace pointing directly into shared
> memory thus saving a memory copy. It may not sound like a big deal,
> but with many HD feeds running at the same time, it becomes critical
> for high performance as a computers memory transfer capacity is
> limited, no matter how many cores you put into it. Even if the color
> space conversion module was left out, it would be nice to be able to
> do something like
> 
>   gst-launch  some_video_src ! ffdec_h264 ! shmsink
>   gst-launch  some_video_src ! ffdec_h264 ! queue ! shmsink
> 
> 
> and pass a buffer from shmsink to the decoder. In this example the
> decoder is ffdec_h264, but it could be decodebin or another. I realize
> that it would be extra challenge, as it might need an hack of not only
> the module, but also the decoder the module is based on as it would be
> needed that the internal decoder uses an externally specified buffer
> area for its decoded frame. In the example with the queue inbetween,
> it would be optimal, if the queue module receives the buffers from
> shmsink point to its shared memory and not only uses them, but also
> pass them along upstream.
> 
> 
> For the upstream, the scenario is a little different.. Here you could
> have something like
> 
> 
>   gst-launch shmsrc ! ffmpegcolorspace ! x264enc
>   gst-launch shmsrc ! queue ! x264enc
> 
> 
> The example is without parameters. Here the colour space module or the
> queue module would need to signal upstream, when the buffer has been
> used and discarded .... and it this case, it would need to discard the
> buffer in a sensible manner.
> 
> 
> 
> I realize this is all a very special case, but as we get more and more
> HD video, this is becoming important as use of GStreamer for high end
> video systems are growing. Imagine also the savings when jumping from
> 1080p to 4k videos. Now mainstream 4k video is still in the future,
> but it will come.
> 
> 
> Feel free to also take a look at
> http://lists.freedesktop.org/archives/gstreamer-devel/2013-July/041987.html
> 
> 
> 
> Best regards
> 
> Peter Maersk-Moller
> 
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel