[Bug 768079] waylandsink: add support wayland presentation time interface

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=768079

--- Comment #2 from Nicolas Dufresne (stormer) <nicolas at ndufresne.ca> ---
I'm gathering code for that in a branch (careful, this branch is rebased
often).

https://git.collabora.com/cgit/user/nicolas/gst-plugins-bad.git/log/?h=wayland-presentation

Arun and I have been tracing the behaviour without any custom synchronization.
So far we got interesting results. Here's a very meaningful graph

  http://imgur.com/a/7VijX

On this graph, we try to play 30fps video on an output at some unprecise 60Hz.
We can observe that the refresh rate is in fact slower, hence it runs later and
later until the submission hits the previous vblank. So most frame are
displayed on 2 blank, and once in a while it only renders on 1 blank.

This zig-zag result is exactly what we expect and the behaviour is acceptable.
What is not though is the jumps you may notice close to the edges, or whenever
the scheduler decides to kick at the wrong moment. This is noticeable in this
graph by those vertical line that looks like glitches. For that reason, we came
to the same conclusion as George had previously experiment in it's demo branch.

 
https://git.collabora.com/cgit/user/gkiagia/gst-plugins-bad.git/commit/?h=demo

To ensure smoothness, we need to control the submission time to ensure we do
actually chose a vblank instead of leaving it to our luck. The simplest
mechanism is to make sure the presented callback keep triggering and to only
draw on this callback. This callback represent the sooner point it time for
which client will hit the next blank. This is though not the most efficient, so
a step up would be to predict the appropriate presented callback for the
current frame base on the last presented callback and the refresh (or a refresh
calculated from the sequence number if you are precision hungry).

>From there, we'll still be displaying the frame a bit far from the presentation
time. That's because there is a certain latency possibly introduced by the
compositor and/or the driver/hw. We don't know this latency in advance, so we
have no choice but to figure-out at run time. I'll be experimenting few
estimation method, ideally a method that gives a value out of the preroll
phase, and then could suggest the app to update the latency later when a more
accurate value is available. I'll also try and avoid implementing our own sync
for now, but try and be smarter then just replacing GstBuffer pointer like we
do.

If someone knows if there is standard for this. As you can see on the graph, it
produce a zigzag when the video rate is different from the display rate. We can
adjust latency to center this zig-zag around 0, or we can lower that zigzag so
the smallest delay tends to zero. I'm not sure what is best, my intuition tells
me that using center is more accurate even if it means that at some moment the
frames are displayed too soon, and other too late.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.