[Bug 611157] video: API to signal stereoscopic and multiview video

[GStreamer (bugzilla.gnome.org)]

https://bugzilla.gnome.org/show_bug.cgi?id=611157
  GStreamer | gst-plugins-base | unspecified

--- Comment #82 from Gwenole Beauchesne <gb.devel at gmail.com> 2014-04-29 08:10:14 UTC ---
Some more information to consider, and that would be my minimal requirements
actually:

1. It is important to distinguish between view order index and view id. Reason:
view order index determines the decoding/output order of the views within an
access unit. The view id can be anything that the user/encoder generated. So,
we normally always get view indices in increasing order: 0, 1, 2, etc. However,
the view id can be anything: 2, 0, 1 for the previous AU, 1, 2, 0 for the next
AU, etc. 

I suggest we have an id that maps to the view_id (or layer_id), and a flag that
marks the start of an access unit. We could have a voc (view order count) field
instead, but a flag marking the start of the access unit would be just fine
IMHO. I am not inspired today, so my current name suggestion for that flag
would be: "FFB" = "First Frame in Bundle", or "BFF": "Bundle First
Field/Frame". bundle == sequence of views with same PTS.

Why not use PTS change to detect new access unit (bundle) boundaries? Well, PTS
is not accurate and there could be cases where it is never defined(?).

2. Add a means to query downstream that it will actually care about
"multi-views" and "stereo3d" buffers. Reason: we don't need to decode all views
or layers if we are not going to use them. e.g. it would be enough to stick to
the base view/layer.

However, because of (1), this means that we are not always going to display
view0, but simply the first decoded view in the access unit, i.e. the one
really matching the base stream.

The other benefit is that, if downstream doesn't care of extra views, we can
avoid a processing step to compose an S3D buffer.

Anyway, in vaapi, I currently handle vid/voc (view_id / view_order_index)
internally, and real view_id and a flag for the plug-in element layer, stuffed
into the vaapi meta for now.

-- 
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.