[RFC v2] Wayland presentation extension (video protocol)

[Jason Ekstrand]

On Mon, Feb 10, 2014 at 5:51 AM, Pekka Paalanen <ppaalanen at gmail.com> wrote:

> On Sat, 8 Feb 2014 15:39:57 -0600
> Jason Ekstrand <jason at jlekstrand.net> wrote:
>
> > On Wed, Feb 5, 2014 at 1:32 AM, Pekka Paalanen <ppaalanen at gmail.com>
> wrote:
> >
> > > On Thu, 30 Jan 2014 17:35:17 +0200
> > > Pekka Paalanen <ppaalanen at gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > it's time for a take two on the Wayland presentation extension.
> > > >
> > > >
> > > >               1. Introduction
> > > >
> > > > The v1 proposal is here:
> > > >
> > >
> http://lists.freedesktop.org/archives/wayland-devel/2013-October/011496.html
> > > >
> > > > In v2 the basic idea is the same: you can queue frames with a
> > > > target presentation time, and you can get accurate presentation
> > > > feedback. All the details are new, though. The re-design started
> > > > from the wish to handle resizing better, preferably without
> > > > clearing the buffer queue.
> > > ...
> > > >   <interface name="presentation" version="1">
> > > >     <request name="destroy" type="destructor">
> > > >
> > > >     <request name="feedback">
> > > >     <request name="queue">
> > > >     <request name="discard_queue">
> > > >
> > > >     <event name="clock_id">
> > > >
> > > >   </interface>
> > > >
> > > >   <interface name="presentation_feedback" version="1">
> > > >     <request name="destroy" type="destructor">
> > > >
> > > >     <event name="sync_output">
> > > >     <event name="presented">
> > > >     <event name="discarded">
> > > >   </interface>
> > >
> > > Hi,
> > >
> > > another random thought; should we support queueing frames (buffers) in
> > > reverse chronological order?
> > >
> > > It's not really possible with the scheduling algorithm I wrote down in
> > > the spec. There is no timestamp associated with the currently showing
> > > content, which means that if you queue a frame with a target timestamp
> > > in the past, it will replace the current content, even if the current
> > > content was originally queued with a later target timestamp.
> > >
> > > I wonder if we could define, that the current content effectively has
> > > the timestamp of when it was presented, and all queued updates with
> > > an earlier target timestamp will be discarded.
> > >
> > > That should work, right?
> > >
> > > Now, is there a corner case... output update has been submitted to
> > > hardware but has not been presented yet, which means the content in
> > > flight has no timestamp determined yet... but we won't update the
> > > output again before the update in flight has completed, which gives the
> > > presented timestamp for the was-in-flight current content.
> > >
> > > If we do need the timestamp for content in flight, we could use the
> > > target timestamp it had when queued, or the timestamp the compositor is
> > > targeting. Since clients have a choice between queued and immediate
> > > updates, I guess using the compositor's target timestamp would be
> > > better defined.
> > >
> > > Opinions?
> > >
> >
> > I agree.  The current frame (or frame for the currently pending flip)
> > should be treated as if it has a timestamp of its expected next
> > presentation time.  I'm still not completely understanding the algorithm
> > for presenting stuff correctly, but it should be possible for the client
> to
> > sneak a frame in for the next present.
> >
> > I need some time at my chalkboard to try and get my head wrapped around
> all
> > this.  Maybe it would be good if you made a couple of little timeline
> > pictures to go with it?
>
> Hi,
>
> I took the advice and drew a picture, attached. The picture is only
> about choosing the right update from the queue for the coming output
> refresh, and ignores the surface's existing content.
>

Thank you!  A picture really is worth 1000 words and, in my experience, an
equation is worth about 200 or so.  That helps clear things up a lot.


>
> A compositor has a queue with updates T1..T5. The current time is
> indicated in the picture, and the next output refresh will be P4. From
> the queue, T4 will be selected, and T1, T2, and T3 are discarded. Both
> T3 and T4 target the same flip, so the one with the latter timestamp
> wins. T5 is too far in the future.
>
> The decision boundaries are at the middle point between the two refresh
> times.
>
> I drew this without rechecking my spec wording, this is what I meant
> there IIRC.
>
> The already presented content update's timestamp can only be in the
> past, so it does not affect the drawn situation.


Good point, I think this is key.



> However, if the queue
> contained only T2, and the current content was presented at P2, then T2
> would be presented at P4. This could happen if the client queues T2
> after P3 has passed[1]. OTOH, if the queue had only T2 and the current
> content was at P3, then T2 should be discarded. Right? This is what I
> need to fix in the spec wording.
>

Yeah, that make sense.


>
> I hope this clarifies the algorithm. Does it make sense?
>
> This algorithm aims to start showing an update between t-T/2 and t+T/2,
> which means that presentation may occur a little early or late, with
> an "average" of zero. Another option would be to show the update between
> t and t+T, which would mean that presentation would be always late with
> an "average" of T/2.
>
> I would assume that zero mean would be "better", but this investigation
> does not consider the jitter when trying present at a constant
> frequency that may or may not be the same as the output refresh
> frequency.
>

Honestly, I don't think it matters for the most part.  Clients will already
be targetting flips and will choose their timestamps based on some
algorithm to try and hit those flips.  Given this algorithm, they'll pick
the flip timestamp;  other algorithms may drive them to request
presentation a little early or late to hit the flip they want.  As long as
it's consistent across compositors, I think we'll be fine.


>
> Once we get all this implemented, we can tune the algorithm based on
> experimental data. I also expect that Weston may need to start
> delaying its repaints rather than repaint immediately after
> finish_frame, for clients that use immediate commits rather than queued
> updates.
>

Agreed.  Unless someone else has comments, it sounds like it's time to play
with it and see what happens.


>
> Another question is, how specifically should the algorithm be described
> in the protocol specification to not restrict future development.
>

I'll think on that.


>
>
> [1] If you think of this case further, you see what would happen if the
> client started queueing frames in reverse-chronological order: no
> matter what phase the compositor is in and how it races with the client,
> the compositor would pick the earliest frame it can without jumping
> the animation backwards in time. Clients could use this to re-start an
> animation without waiting for a roundtrip for the server to ack
> presentation.discard_queue, and deducing which buffer is currently
> presenting. Though, I'm not sure what a realistic use case would be...
>
>
> Thank


Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/wayland-devel/attachments/20140210/0c0e15e8/attachment-0001.html>