[Bug 611157] [RFC] more buffer flags and caps fields in gst-video for 3d video

[GStreamer (bugzilla.gnome.org)]

https://bugzilla.gnome.org/show_bug.cgi?id=611157
  GStreamer | gst-plugins-base | unspecified

--- Comment #19 from sreerenj <bsreerenj at gmail.com> 2013-03-24 10:02:27 UTC ---
I would like to share some details about the StereoscopicContents available:
(This is my current understanding.. :))

So far I have seen two types of stereo videos: (for explanation I am taking
h264 as stream and mp4 as container format)

type1 : This stereo video content storage is based on MPEG Application format.
To support this in gstreamer we doesn’t need to change anything in the parser
or decoder. Stereo video would be either packed format (two views in a single
frame) or sequential frames (only progressive).  It supports 2 types of packed
formats: 1) side_by_side 2)vertical_interleaved..Here left and right views are
just like regular frames. But packed as single frame. This is  *NOT* using any
h264-stereoscopic extension or MultiView extension to encode the video. Instead
each of the two views are packed and encoded as a single frame (using regular
h264 profiles..). So for supporting this (i am using qtdemux as example):
  -- We need to parse the GstVideoStereoType in qtdemux.
  -- We can also parse the GstVideoStereoContentType in qtdemux.

type2: This type is based on the the Annex-H extension of h264-spec .More
optimized stereo encoding based on StereoScopic profiles.Unfortunately I
haven’t seen these type video samples in any containers :)..We can parse the 
GstVideoStereoType from qtdemux. But we will get the GstVideoStereoContentType
only during stream parsing (SEI palyloads), either from h264parse or from
decoders supporting it... There are raw h264 samples available in web which is
encoded with this extension and all of them are sequential_view_streams: means,
"first_frame_is_left_view/second_frame_is_right_view"  or
"first_frame_is_right_view/second_frame_is_left_view" type...As per
FramePackingArrangeMent info in h264 SEI payload, if the stream type is
GST_VIDEO_STEREO_PACKED_ROW_INTERLEAVED,
GST_VIDEO_STEREO_PACKED_COLUMN_INTERLEAVED,
  GST_VIDEO_STEREO_PACKED_SIDE_BY_SIDE,
  GST_VIDEO_STEREO_PACKED_TOP_BOTTOM,
or   GST_VIDEO_STEREO_PACKED_CHECK_BOARD_INTERLEAVED
we need to do some upsampling for each view based on  Figure D-1 to D-11
of H264_spec. Because each view has SAR which is equal to that the SAR of
frame.

Initially we can support SIDE-BY-SIDE_packed and sequential streams.

Need more efforts:
For handling GST_VIDEO_STEREO_PACKED_COLUMN_INTERLEAVED, we need some
vertical_line_based de-interlacing method/or some elements to do that??...
Likewise for GST_VIDEO_STEREO_PACKED_CHECK_BOARD_INTERLEAVED , we need
quincunx_sampling.

-- 
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.