[RFC v2] Wayland presentation extension (video protocol)

[Pekka Paalanen]

On Sat, 8 Feb 2014 15:39:57 -0600
Jason Ekstrand <jason at jlekstrand.net> wrote:

> On Wed, Feb 5, 2014 at 1:32 AM, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> 
> > On Thu, 30 Jan 2014 17:35:17 +0200
> > Pekka Paalanen <ppaalanen at gmail.com> wrote:
> >
> > > Hi,
> > >
> > > it's time for a take two on the Wayland presentation extension.
> > >
> > >
> > >               1. Introduction
> > >
> > > The v1 proposal is here:
> > >
> > http://lists.freedesktop.org/archives/wayland-devel/2013-October/011496.html
> > >
> > > In v2 the basic idea is the same: you can queue frames with a
> > > target presentation time, and you can get accurate presentation
> > > feedback. All the details are new, though. The re-design started
> > > from the wish to handle resizing better, preferably without
> > > clearing the buffer queue.
> > ...
> > >   <interface name="presentation" version="1">
> > >     <request name="destroy" type="destructor">
> > >
> > >     <request name="feedback">
> > >     <request name="queue">
> > >     <request name="discard_queue">
> > >
> > >     <event name="clock_id">
> > >
> > >   </interface>
> > >
> > >   <interface name="presentation_feedback" version="1">
> > >     <request name="destroy" type="destructor">
> > >
> > >     <event name="sync_output">
> > >     <event name="presented">
> > >     <event name="discarded">
> > >   </interface>
> >
> > Hi,
> >
> > another random thought; should we support queueing frames (buffers) in
> > reverse chronological order?
> >
> > It's not really possible with the scheduling algorithm I wrote down in
> > the spec. There is no timestamp associated with the currently showing
> > content, which means that if you queue a frame with a target timestamp
> > in the past, it will replace the current content, even if the current
> > content was originally queued with a later target timestamp.
> >
> > I wonder if we could define, that the current content effectively has
> > the timestamp of when it was presented, and all queued updates with
> > an earlier target timestamp will be discarded.
> >
> > That should work, right?
> >
> > Now, is there a corner case... output update has been submitted to
> > hardware but has not been presented yet, which means the content in
> > flight has no timestamp determined yet... but we won't update the
> > output again before the update in flight has completed, which gives the
> > presented timestamp for the was-in-flight current content.
> >
> > If we do need the timestamp for content in flight, we could use the
> > target timestamp it had when queued, or the timestamp the compositor is
> > targeting. Since clients have a choice between queued and immediate
> > updates, I guess using the compositor's target timestamp would be
> > better defined.
> >
> > Opinions?
> >
> 
> I agree.  The current frame (or frame for the currently pending flip)
> should be treated as if it has a timestamp of its expected next
> presentation time.  I'm still not completely understanding the algorithm
> for presenting stuff correctly, but it should be possible for the client to
> sneak a frame in for the next present.
> 
> I need some time at my chalkboard to try and get my head wrapped around all
> this.  Maybe it would be good if you made a couple of little timeline
> pictures to go with it?

Hi,

I took the advice and drew a picture, attached. The picture is only
about choosing the right update from the queue for the coming output
refresh, and ignores the surface's existing content.

A compositor has a queue with updates T1..T5. The current time is
indicated in the picture, and the next output refresh will be P4. From
the queue, T4 will be selected, and T1, T2, and T3 are discarded. Both
T3 and T4 target the same flip, so the one with the latter timestamp
wins. T5 is too far in the future.

The decision boundaries are at the middle point between the two refresh
times.

I drew this without rechecking my spec wording, this is what I meant
there IIRC.

The already presented content update's timestamp can only be in the
past, so it does not affect the drawn situation. However, if the queue
contained only T2, and the current content was presented at P2, then T2
would be presented at P4. This could happen if the client queues T2
after P3 has passed[1]. OTOH, if the queue had only T2 and the current
content was at P3, then T2 should be discarded. Right? This is what I
need to fix in the spec wording.

I hope this clarifies the algorithm. Does it make sense?

This algorithm aims to start showing an update between t-T/2 and t+T/2,
which means that presentation may occur a little early or late, with
an "average" of zero. Another option would be to show the update between
t and t+T, which would mean that presentation would be always late with
an "average" of T/2.

I would assume that zero mean would be "better", but this investigation
does not consider the jitter when trying present at a constant
frequency that may or may not be the same as the output refresh
frequency.

Once we get all this implemented, we can tune the algorithm based on
experimental data. I also expect that Weston may need to start
delaying its repaints rather than repaint immediately after
finish_frame, for clients that use immediate commits rather than queued
updates.

Another question is, how specifically should the algorithm be described
in the protocol specification to not restrict future development.


[1] If you think of this case further, you see what would happen if the
client started queueing frames in reverse-chronological order: no
matter what phase the compositor is in and how it races with the client,
the compositor would pick the earliest frame it can without jumping
the animation backwards in time. Clients could use this to re-start an
animation without waiting for a roundtrip for the server to ack
presentation.discard_queue, and deducing which buffer is currently
presenting. Though, I'm not sure what a realistic use case would be...


Thanks,
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: frame_queue_scheduling.png
Type: image/png
Size: 30339 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/wayland-devel/attachments/20140210/664e719e/attachment-0001.png>