glxsync - explicit frame synchronization sample implementation

Tue Jan 4 09:56:37 UTC 2022

On 12/30/21 13:19, Michael Clark wrote:
> On 12/30/21 11:19 PM, Eero Tamminen wrote:
>> Hi,
>>
>> Different window managers do resizes differently.  See for example:
>>      https://gitlab.gnome.org/GNOME/mutter/-/issues/60
>>
>> On which window managers did you test this?
> 
> I did mention that. I tested with the mutter compositor inside 
> gnome-shell, along with whichever decorator extensions are loaded from 
> the toolkits present in GNOME on Ubuntu 20.04 LTS and Ubuntu 21.10.
> 
> ; ( for i in $(pgrep gnome-shell); do sudo lsof -p $i ; done ) | grep 
> mutter | awk '{ print $9; }'
> 
> /usr/lib/x86_64-linux-gnu/mutter-6/Meta-6.typelib
> /usr/lib/x86_64-linux-gnu/mutter-6/ClutterX11-6.typelib
> /usr/lib/x86_64-linux-gnu/mutter-6/Cogl-6.typelib
> /usr/lib/x86_64-linux-gnu/mutter-6/Clutter-6.typelib
> /usr/lib/x86_64-linux-gnu/mutter-6/Cally-6.typelib
> /usr/lib/x86_64-linux-gnu/mutter-6/CoglPango-6.typelib
> /usr/lib/x86_64-linux-gnu/mutter-6/libmutter-cogl-path-6.so.0.0.0
> /usr/lib/x86_64-linux-gnu/mutter-6/libmutter-cogl-6.so.0.0.0
> /usr/lib/x86_64-linux-gnu/libmutter-6.so.0.0.0
> 
> One would hope folks implement standards in such a way that clients 
> using the documented window manager sync protocols are interoperable.
> 
> This is in fact the reasoning that spurred this email. Is there a 
> reference implementation for frame synchronization for X11 GLX apps?
> 
> I guess I am primarily satisfied if I can solve the artifacts on the 
> window manager that I use but if I am distributing an app more widely 
> then I might like something well-tested and a little more robust.
> 
> Does KDE implement _NET_WM_SYNC_REQUEST et al?

Yes, kwin supports _NET_WM_SYNC_REQUEST, it doesn't support the extended 
protocol version though.

Regards,
Vlad

>>      - Eero
>>
>> On 30.12.2021 7.20, Michael Clark wrote:
>>> Dear Mesa Developers,
>>>
>>> I have been using GLFW for tiny cross-platform OpenGL demos for some 
>>> time but something that has really been bothering me are the visual 
>>> artifacts when resizing windows. Over the last year or so I have made 
>>> multiple attempts at solving this issue, digging progressively deeper 
>>> each time, until spending the last month researching compositor 
>>> synchronization protocols, reading compositor code, and writing this 
>>> demo as a prelude to figuring out how one might fix this issue in 
>>> GLFW or even Chrome.
>>>
>>> I decided that first it might be a good idea to come up with the 
>>> simplest possible isolated example comprising of a near complete 
>>> solution without the unnecessary complexity of layering for all of 
>>> the cross-platform abstractions. It seems to me despite the ease this 
>>> can be solved with Wayland EGL, it is still useful, primarily for 
>>> wider compatibility, to be able to package X11 GLX applications, 
>>> which is the window system that I typically use when targeting Linux 
>>> with GLFW.
>>>
>>> That brings me to _glxsync_ which is an attempt at creating a 
>>> minimally correct implementation of explicit frame synchronization 
>>> using X11, GLX, XSync and the latest compositor synchronization 
>>> protocols [1,2], tested to work with mutter and GNOME on Xorg or 
>>> Xwayland.
> 
> ^^ here
> 
>>> - https://github.com/michaeljclark/glxsync/
>>>
>>> _glxsync_ is an X Windows OpenGL demo app using GLX and XSync 
>>> extended frame synchronization responding to synchronization requests 
>>> from the compositor in response to configuration changes for window 
>>> resizes. The demo updates extended synchronization counters before 
>>> and after frames to signal to the compositor that rendering is in 
>>> progress so that buffers read by the compositor are complete and 
>>> matches the size in configuration change events. It also has 
>>> rudimentary congestion control.
>>>
>>> _glxsync_ depends on the following X11 window system atoms:
>>>
>>> - _NET_WM_SYNC_REQUEST
>>> - _NET_WM_SYNC_REQUEST_COUNTER
>>> - _NET_WM_FRAME_DRAWN
>>> - _NET_WM_FRAME_TIMINGS
>>> - _NET_WM_PING
>>>
>>> _glxsync_ *does not* yet implement the following extensions:
>>>
>>> - _NET_WM_SYNC_FENCES
>>> - _NET_WM_MOVERESIZE
>>>
>>> _glxsync_ depends on the following libraries: _X11, Xext, GLX, GL_.
>>>
>>> I have to say there were numerous subtle issues that I found while 
>>> testing this code on Ubuntu 21.10 XWayland with an Intel Mesa 
>>> graphics stack and Ubuntu 20.04 LTS Xorg with the NVIDIA proprietary 
>>> graphics stack, so I have no idea how it will fly with other drivers 
>>> and am very interested in feedback. There really is not much sample 
>>> code that I could find that addresses this issue.
>>>
>>> I found the Intel driver particularly finicky and there are some very 
>>> carefully placed XFlush calls *before* frame renders, and XSync calls 
>>> during congestion. There are also the beginnings of adaptive frame 
>>> rate using frame times and render timings stored in a circular 
>>> buffer. That said, there is no advanced adaptive frame rate logic 
>>> beyond detecting circumstances that can lead to tears with a back-off 
>>> to the measured short term average frame rate from statistics, and 
>>> some logic to delay frames when there are collisions with Expose events.
>>>
>>> There is also some rudimentary tracing infrastructure and some 
>>> carefully placed calls to poll, XEventsQueued(d, QueuedAlready), 
>>> XEventsQueued(d, QueuedAfterReading) to avoid blocking in XNextEvent 
>>> at all costs. I found it necessary to add a heuristic to avoid frame 
>>> submission until receiving frame timings from the compositor. 
>>> Intuitively one might think this makes the loop synchronous, but with 
>>> the NVIDIA driver, it appears the heuristic still allows multiple 
>>> frames to be submitted in advance. It is certainly finicky to debug. 
>>> There is a --no-sync option to simulate the absence of compositor 
>>> synchronization as a testing aid.
>>>
>>> There is very little back-pressure signaling to the client beyond the 
>>> ability to observe timings and serial numbers in frame drawn and 
>>> frame timing messages. It worries me that I need very careful 
>>> placement of XFlush and XSync to make the demo work so I would really 
>>> appreciate feedback if I am doing it wrong. There is some interesting 
>>> potential for control loops when using stats for adaptive frame rate, 
>>> so I have not yet attempted any sophisticated congestion control 
>>> algorithm.
>>>
>>> In any case I am sharing this code with the hopes that folk can help 
>>> with testing. I was thinking to make a patch for GLFW but this was a 
>>> first step. I would really appreciate if folks could help test on 
>>> different drivers such as nouveau and amdgpu as I don't have access 
>>> to them. The code is currently released under the PLEASE LICENSE 
>>> which is practically public domain with one exception, but I am not 
>>> disinclined towards releasing it under an MIT license if it were 
>>> found to be a useful sample to add to the mesa demos.
>>>
>>> Is there a place in mesa-demos for a frame synchronization demo? I 
>>> see glsync. Is there a compositor sync example that I may have 
>>> missed? I can imagine with the addition of WM_MOVERESIZE it could be 
>>> used for tests. This is pretty much version 0.0.1. i.e. is clean 
>>> enough to release.
>>>
>>> Regards,
>>> Michael Clark
>>>
>>> [1] https://fishsoup.net/misc/wm-spec-synchronization.html
>>> [2] https://lwn.net/Articles/814587/