[gst-devel] Video 3D support

[Stefan Kost]

On 23.05.2010 07:47, David Schleef wrote:
> On Tue, May 18, 2010 at 07:12:26AM +0000, Martin Bisson wrote:
>   
>> My work is based on the proposal made for Google Summer of Code :
>> http://gstreamer.freedesktop.org/wiki/Video3DSupport , and I would like to
>> discuss the issue discussed in
>> https://bugzilla.gnome.org/show_bug.cgi?id=611157.  I'm actually trying to
>> get input on what needs to be added to GStreamer in order to provide proper
>> 3D video support, by adding info to the 3D stream.  The different options
>> could be any combination of:
>>
>> 1) doing nothing : the resulting stream would just be treated as a "normal"
>> video stream
>> 2) adding caps : the caps could have information about the 3D video
>> information (left-right, top-bottom, red-green, etc.)
>> 3) adding buffer flags : the info would be in the buffer flags, like audio
>> streams (number of "channels" (left/right video streams seen as left/right
>> sound channels), etc.)
>>     
> The goal, imo, is to define how GStreamer handles stereo video
> *natively*, that is, elements can automatically differentiate
> between normal and stereo video, caps negotiation works as one
> would expect, playback of a stereo video on a normal monitor
> would allow for the option of viewing in mono or with red/green
> glasses, etc.  A lower goal would be to create elements that
> manipulate stereo video, but entirely manually.  I'm only
> concerned with the former.
>   
Ack. That should be the scope of the project too. Thats why we'd like to
get feedback on caps/flags.
> There are two main options: Use pairs of well-known raw image
> layouts (i.e., I420, YUY2, UYVY, etc.), probably consecutive
> in memory, for the left and right images.  Or, define a bunch
> of new fourccs that mean "stereo video" that correspond to
> existing layouts, but enhanced for stereo.
>
> Using existing layouts: I recommend packing the two pictures
> consecutively in memory, left then right.  The main rationale is
> that conversion to a normal picture is simply changing the size
> and/or pointer of the buffer and changing caps.  Other packing
> arrangements might be important in the future, so having a
> manditory caps field marking the packing would be a good idea.
>   
yep. over/under would mean left than right. We should support left/right
for interoperability, but its slower to process.
> I recommend using a new caps type, perhaps video/x-raw-yuv-stereo
> or some such, instead of using video/x-raw-yuv,stereo=true.  Using
> video/x-raw-yuv leads to endless compatibility problems: elements
> that currently handle video/x-raw-yuv would silently do the wrong
> thing with stereo video.  Using x-raw-yuv would mean width/height
> in the caps would be double the *actual* width/height of the mono
> video, which is hacky.  Also, converting from stereo to mono in
> many cases would require copying.
>   
We should not change the semantics of width/height, so if a plugin is
not knowing about stereo=true, it would process only first half of the
buffer, that is the left frame. If the element is inplace, the 2nd half
is not modified. If it is creating a new buffer, we loose the
stereo=true. For left/right packing we would get garbage :/

> Defining new fourccs:  This has the obvious disadvantage that
> we'd either need to keep these internal to GStreamer, or make them
> well-known enough for other people to use them.  Integrating with
> existing elements (and libgstvideo) is straightfoward.  Adding
> new layouts (side-by-side, top-bottom, memory consecutive, etc.)
> is simple, although adds *lots* more fourccs.
>
> I think that overall, using new fourccs would involve writing
> less code and be less prone to bugs.  It is my preference.
>   
I Honestly don't like it so much as its badly scales :/ To be sure you
mean instead of
video/x-raw-yuv, format="I420" we do video/x-raw-yuv,
format="S420",layout="over/under" (yeah crap). Or
video/x-raw-yuv-stereo, format="I420",layout="over/under" ?

> Oh yeah, other methods such as dual pads for left/right, or buffer
> flags, are non-starters: our attempts at dual pads for float audio
> failed miserably, and we don't have any buffer flags available.
> And there are other reasons which I don't feel like enumerating
> right now.
>
>   
Don't worry, I remmeber some issue with these :)

Stefan
>
> dave...
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>