[gst-devel] New buffer metadata and caps propositions and stereo video

[Martin Bisson]

Hi,

I worked on stereo video during a Google Summer of Code project this 
summer, and for the time being it's in a separate branch on Gitorious 
(http://gitorious.org/video3d/video3d), and I wanted to know what was 
needed in order to integrate it with the main GStreamer tree.

Again, to be clear, my main objective is to do what needs to be done in 
order to integrate the stereo plugins in GStreamer.

In order to do so, the main point of discussion is about how to "label" 
the stereo buffers.  There seems to be discussions right now on adding 
meta-data to buffers (see 
http://cgit.freedesktop.org/~wtay/gstreamer/tree/docs/design/draft-buffer2.txt) 
and on new caps for buffers with additional info 
(http://gstreamer.freedesktop.org/wiki/NewGstVideo).  There was also 
talks about buffer flags.  Stereo video obviously has to fit somewhere 
in there.  And these seem to be currently discussed issues.

One issue I see with stereo information is that there are some cases 
where we would like images in a stereo buffer to be treated as a normal 
video stream by existing plugins (to apply color correction, gain or 
whatever), but there are cases where we would want caps negotiation to 
fail if you're trying to plug a normal stream in a stereo sink pad and 
vice-versa.

One way to achieve that would be to have 2 kinds of stereo information.  
The first one is to create a new set of caps and is the one that I've 
used my stereo branch (for instance video/x-raw-rgb-stereo).  That way, 
plugins such as the anaglyph image generator can make sure they get 
stereo streams.  We could also have a second way to add stereo 
information, which would be either to use buffer flags or buffer 
meta-data or something along those lines that is NOT involved in caps 
negotiation.  That way, a buffer could be tagged with "left view" or 
"right view" and could still be processed as a normal buffer by existing 
plugins.  We would only need a plugin that could take a "real stereo" 
stream with "real" stereo caps (such as video/x-raw-rgb-stereo), and 
convert it to a "normal" stream (which caps such as video/x-raw-rgb) but 
with each buffer tagged with "left" or "right" tag.  And another plugin 
to do the opposite.

Right now, I only implemented the use of new caps 
(video/x-raw-rgb-stereo) in the stereo branch, because the 
meta-data/buffer flags/etc. seems to be going to change so...

That's my take on it.  I'm obviously very eager to hear what you might 
have to say about it and what you would like me to do in order to make 
my branch "GStreamer-ready".

Thanks,

Martin