Wayland presentation extension (video protocol) WIP after RFCv2

[Daniel Stone]

Hi Michel,

On 13 April 2018 at 10:17, Michel Dänzer <michel at daenzer.net> wrote:
> reviving an old topic... Inspired by the thread "RFC for a render API to
> support adaptive sync and VRR" currently happening on the dri-devel list.

I've been following that thread pretty keenly.

> On 2014-02-26 10:30 AM, Pekka Paalanen wrote:
>> Since then, there has been plenty of discussion, some protocol changes
>> towards a v3, and I have a WIP implementation. My current work can be
>> found at
>> http://cgit.collabora.com/git/user/pq/weston.git/log/?h=presentation-WIP2
>> for now. When I get back, the branch will be rewritten and/or
>> abandoned.
>
> I looked at
> https://git.collabora.com/cgit/user/pq/weston.git/commit/?h=presentation-WIP4&id=5cacb71c28052d9a491ad969d74d9145cfe9936d

FWIW, the feedback - not queuing - side got merged a long time ago:
https://cgit.freedesktop.org/wayland/wayland-protocols/tree/stable/presentation-time/presentation-time.xml

> First, I'd like to question that "turns-to-light" are good semantics for
> the timestamps, since the Wayland compositor and driver stack generally
> don't know the delay from scanout to the display emitting light.

I agree. In reality, it's implemented as the timestamp we got from
KMS, which a) is not turned-to-light, and b) not directly useful to
the app. Knowing what I do now, I'd probably want:
  * the compositor's critical decision point, i.e. the absolute latest
time at which the client must have committed its buffer and have the
commit processed by the compositor (depends on compositor
implementation)
  * the hardware/driver's critical decision point, i.e. when all
fences must have signalled in order to present for the next frame
(just before start of vbl / end of previous vactive; strictly later
than previous)
  * actual real turned-to-light (strictly later than previous and
almost certainly static offset gained from CEA or whatever)

> Applications which are sensitive to this delay can deal with it by other
> means, e.g.:
>
> * Video players usually have a delay setting, which the user can tweak
>   for optimal audio/video sync.
> * Some games have a calibration function, e.g. displaying regular
>   flashes and asking the player to press a button on each flash.
> * Mario has access to very precise measurement gear at work, the data
>   from which is used in applications for this I presume.
>
> The protocol tries to account for that the compositor may approximate
> the "turns-to-light" time:
>
>         Compositors may approximate this from the framebuffer flip
>         completion events from the system, and the latency of the
>         physical display path if known.

You can probably tell from this that it was mainly designed for media
A/V sync, and not necessarily to drive repaint loops.

> But I'm afraid that's not really helpful, as it means the application
> cannot trust the timestamps but needs something as described above
> anyway. So it doesn't really buy anything over "scanout starts"
> semantics, for which the driver stack can provide accurate timestamps.

If we know from CEA that we have a fixed 100ms latency on the output
path, we should report that to the client. We're not going to get it
perfect obviously, but I think getting as accurate time-to-light as we
possibly can is a really good goal. It just doesn't solve the other
two cases, for the winsys client to know when it has to present to the
compositor, and also to know when any fences have to have completed.

> Also, both higher-level APIs I know of which allow the application to
> specify a target timestamp for presentation (VDPAU and
> VK_GOOGLE_display_timing) use "not before" semantics. So, maybe the
> not_before flag can be dropped, and flags can be added later for
> different semantics, if a need for them ever materializes.

... that being said, VK_GOOGLE_display_timing's 'present margin' is
probably enough to cover these, rather than having to deal in absolute
timestamps.

Cheers,
Daniel