Passing buffers upstream

[Peter Maersk-Moller]

Hi.

I wonder if the GStreamer API offers the possibility of a plugin to pass an
allocated buffer upstream? An would it be possible for a plugin to
allocate/construct the buffer from its own memory allocation scheme?
Technically it would be so difficult, if the API has the right hooks. More
difficult would it probably be, if a module would like pass one of its own
allocated buffer downstream, but needs to be told, when the buffer is free
again.

Passing upstream is very useful if the plugin module shmsink could pass a
buffer upstream that points to an area in its shared memory. Likewise the
shmsrc, could pass a buffer downstream pointing to its shared memory. I
understand that the two cases are technical different, especially the
downstream is difficult, as the consumer of the buffer must signal
upstream, that the buffer has been used and are free for reuse by the
originator of the buffer.

Why is this useful? Imagine you have a video mixer (like Snowmix) that
needs its input video feeds from shmsink and imagine there are mutiple
video feeds like 10 feeds and imagine these feeds runs as 1080p at 30fps
and imagine the format is RGBA for optimal quality, then that amounts to
around 245MB per second per feed. That is really a lot of data. Being able
to limit the amount of unnecessary data transfer would really help. Imagine
a pipeline like below

  gst-launch  some_video_src ! ffdec_h264 ! ffmpegcolorspace ! shmsink

Parameters are left out in this example and the example uses gst version
0.10 modules. Version 1.x is slightly different. In the example, the system
would save a memory transfer if the shmsink module could pass a buffer to
ffmpegcolorspace pointing directly into shared memory thus saving a memory
copy. It may not sound like a big deal, but with many HD feeds running at
the same time, it becomes critical for high performance as a computers
memory transfer capacity is limited, no matter how many cores you put into
it. Even if the color space conversion module was left out, it would be
nice to be able to do something like

  gst-launch  some_video_src ! ffdec_h264 ! shmsink
  gst-launch  some_video_src ! ffdec_h264 ! queue ! shmsink

and pass a buffer from shmsink to the decoder. In this example the decoder
is ffdec_h264, but it could be decodebin or another. I realize that it
would be extra challenge, as it might need an hack of not only the module,
but also the decoder the module is based on as it would be needed that the
internal decoder uses an externally specified buffer area for its decoded
frame. In the example with the queue inbetween, it would be optimal, if the
queue module receives the buffers from shmsink point to its shared memory
and not only uses them, but also pass them along upstream.

For the upstream, the scenario is a little different.. Here you could have
something like

  gst-launch shmsrc ! ffmpegcolorspace ! x264enc
  gst-launch shmsrc ! queue ! x264enc

The example is without parameters. Here the colour space module or the
queue module would need to signal upstream, when the buffer has been used
and discarded .... and it this case, it would need to discard the buffer in
a sensible manner.

I realize this is all a very special case, but as we get more and more HD
video, this is becoming important as use of GStreamer for high end video
systems are growing. Imagine also the savings when jumping from 1080p to 4k
videos. Now mainstream 4k video is still in the future, but it will come.

Feel free to also take a look at
http://lists.freedesktop.org/archives/gstreamer-devel/2013-July/041987.html

Best regards
Peter Maersk-Moller
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20130728/d466bfde/attachment.html>