[gst-devel] Video 3D support

[David Schleef]

On Tue, May 25, 2010 at 04:29:39PM +0300, Stefan Kost wrote:
> > I think that overall, using new fourccs would involve writing
> > less code and be less prone to bugs.  It is my preference.
> >   
> I Honestly don't like it so much as its badly scales :/ To be sure you
> mean instead of
> video/x-raw-yuv, format="I420" we do video/x-raw-yuv,
> format="S420",layout="over/under" (yeah crap). Or
> video/x-raw-yuv-stereo, format="I420",layout="over/under" ?

I meant "video/x-raw-yuv,format=S420,width=1280,height=720" for the
native way that GStreamer handled stereo video.  (Funny that you
used S420, as that was exactly the method for mangling fourcc's that
I was thinking: I420 -> S420, UYVY -> SYVY, etc.)

And no "layout" property.  We already have a fourcc to indicate layout
(as well as other stuff, under which rug we now also sweep stereo/mono).

Side-by-side and top-bottom is a misuderstanding of what "native"
means.  These are methods of packing two pictures into a single
picture for the purposes of shoving it through software that only
understands one picture.  We want a system that understands *two*
pictures.

As a data point, H.264 handles stereo by doubling the number of
pictures in the stream and ordering them left/right/left/right.  The
closest match in a GStreamer API would be to use buffer flags, but
that's gross because a) we don't have any buffer flags available
unless we steal them from miniobject, b) we still would need a
field (stereo=true) in the caps, which would cause compatibility
issues, c) some existing elements would work fine (videoscale),
others would fail horribly (videorate).

The second closest match is what I recommended: format=(fourcc)S420
(as above), indicating two I420 pictures consecutive in memory.  A
stereo H.264 decoder can be modified to decode to these buffers
easily.

On the display side, I only really have experience with X output
to shutter stereo goggles using OpenGL: you upload separate pictures
for right and left, and the driver flips between them.  In this
case, the packing in memory is only slightly important -- memory
consective order would be easier to code, but the graphics engine
could easily be programmed to handle top/bottom or side-by-side.

HDMI 1.4, curiously, has support for half a dozen stereo layouts.
This is a design we should strive to avoid.

On the question of "How do I handle side-by-side video":  Use an
element to convert to the native format.  Let's call it
'videotostereo'.  Thus, if you have video like on this page:
http://www.stereomaker.net/sample/index.html, you would use
something like:

  filesrc ! decodebin ! videotostereo method=side-by-side !
    stereoglimagesink

Assuming you don't have stereo output, but want to watch one of
the mono channels:

  filesrc ! decodebin ! videotostereo method=side-by-side !
    ffmpegcolorspace ! xvimagesink

Converting the above video to red/cyan anaglyph:

  filesrc ! decodebin ! videotostereo method=side-by-side !
    videofromstereo method=red-cyan-anaglyph !  xvimagesink



dave...