How to make infinite-scrolling GUI widgets under Wayland ?

[Jonas Ådahl]

On Mon, Apr 26, 2021 at 04:57:01PM +0200, Jean-Michaël Celerier wrote:
> Hello,
> I am working on the multimedia sequencer ossia score (https://ossia.io).
> 
> I am trying to make sure that it works fine for Linux users under wayland.
> 
> For audio (and in general multimedia) apps, there is a very common family
> of GUI widget which allows to scroll a value "infinitely" (or at least much
> more than the height of the screen) - the usual user interaction is :
> * User clicks on the widget : the mouse cursor disappears (some apps will
> give a kind of highlight to the widget to indicate that it is being acted
> upon)
> * User moves their mouse upwards or downwards to increase the value
> * At some point they reach the top or the bottom of the screen : to
> continue scrolling transparently, the app translates the mouse cursor to
> the top of the screen
> * When the user releases the mouse the cursor reappears at the position
> where it was clicked.
> 
> This is needed in two case :
> - To make sliders / knobs with easily adjustable values
> - To implement infinite- or at least very long scrolling in minimaps.
> 
> I made a few videos:
> - Video 1: example of slider with the feature in Ableton Live (one of the
> most used music making app) : https://streamable.com/ecepvc
> 
> - Video 2 : example of minimap in Ableton Live :
> https://streamable.com/epc7r1
> 
> - Video 3 : Example of the pain induced by software that do not support the
> feature: here I want to change the tempo but cannot because the mouse hits
> the top of the screen. Thus I have to release and go click on it again :
> https://streamable.com/bniht5
> 
> Thus, my question is : what must I do as the developer of such an app to
> make sure that my users will have widgets that will work "as expected" for
> people in the media creation field, whatever the wayland compositor my
> users are using (KDE Plasma, GNOME, sway...). I don't want them to have an
> inferior user experience on Linux when compared to other operating systems.
> 
> The googling I've done so far seems to say that there is no way to position
> the cursor absolutely directly through wayland, but the only other way
> seems to be through /dev/uinput which needs root permissions to write to
> (and my userbase are artists who generally don't have the technical
> skillset to know that they must mark things as root; I could always check
> and show a popup that requests the user to do chmod u+rwx on /dev/uinput on
> startup but that would be my last resort...)

To implement the behavior shown in video 1 and 2, the "Wayland" way is
depends on two protocols extension:

 * Pointer locking extension[0]
 * Relative pointer motions extension[1]

With the core wl_pointer protocol object, you can control when the
pointer cursor sprite is showing or not; with the pointer locking
extension, you can effectively lock the pointer position to its
position, meaning it will stay put and won't "escape" your application
window; and lastly, with the relative pointer motions extension, you can
receive pointer motions that are not affected by actual movement of the
pointer cursor sprite, i.e. even if the pointer hit an edge or doesn't
move, you'll still receive the motions as sent from the device.

I think most compositors support these two protocol extensions as they
are a corner stone in how e.g. 3D FPS games work.


Jonas

[0] https://gitlab.freedesktop.org/wayland/wayland-protocols/-/blob/master/unstable/pointer-constraints/pointer-constraints-unstable-v1.xml
[1] https://gitlab.freedesktop.org/wayland/wayland-protocols/-/blob/master/unstable/relative-pointer/relative-pointer-unstable-v1.xml

> 
> Thanks !
> Jean-Michaël

> _______________________________________________
> wayland-devel mailing list
> wayland-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/wayland-devel