RFC: Wayland high-resolution wheel scrolling
Simon Ser
contact at emersion.fr
Fri Apr 5 16:11:24 UTC 2019
On Wednesday, April 3, 2019 5:36 AM, Peter Hutterer <peter.hutterer at who-t.net> wrote:
> I've been trying to sort out the new hi-res wheel scrolling that was added to
> linux 5.0 but it's been a bit of a struggle to say the least. For the
> impatient, skip forward to the protocol diff but the background info is
> important.
Thanks for taking the time to write this big summary. It's pretty helpful!
> Everything below applies equally to horizontal and vertical scrolling, both
> directions and wheels and wheel tilts.
>
> The APIs that are involved for low-resolution wheel scrolling:
>
> kernel:
> - REL_WHEEL for every wheel click
>
> libinput uses pointer axis events for wheels:
> - pointer axis value: the movement of the wheel in degrees
> - pointer axis discrete value: the number of wheel clicks
> - pointer axis source: set to 'wheel'
>
> Usally this means you get discrete 1, value 15 for the standard 15-degree mouse
> wheel, discrete 2 value 30 is two wheel clicks within one hw frame.
>
> There is one implicit assumption here: the discrete value must
> be 1 or more for every event. Otherwise it's impossible to use the value and
> calculate the angle. More on that later.
>
> This API is effectively the same in the wayland protocol except for the unit of
> the 'value' which is is in screen coordinates, not degrees.
>
> wayland protocol:
> - pointer axis value: the movement in screen coordinates
> - pointer axis discrete: the number of wheel clicks
> - pointer axis source: set to 'wheel'
>
> The big difference here though: libinput's API is per-device, wayland's API is
> per wl-seat pointer. There's no way to tell in the protocol that you're
> scrolling simultaneously with wheels on two different mice.
>
> Let's see how the common compositors implement this:
>
> weston:
> - for source wheel, only the discrete value is handled and:
> - pointer axis discrete is sent as-is from libinput
> - pointer axis source is sent as-is from libinput
> - pointer axis value is always (discrete * 10)
> - otherwise the libinput pointer value (degrees) from a source wheel is ignored
>
> The magic multiplier 10 is afaik for historical reasons, weston used to do 10
> so early libinput took that. When libinput switched to use physical dimensions
> instead of 10, that value was kept for backwards compatibility with existing
> wayland clients.
>
> mutter:
> - for source wheel, only the discrete value is handled and:
> - clutter-internal event for the discrete **direction** from libinput
> - i.e. discrete > 1 will still only generate one event,some events
> drop to the floor here for fast wheel movements
> - clutter-internal event with (value * 10) marked as 'emulated' but
> that event is ignored later when we're writing to the protocol
> - the discrete event is written as wl_pointer.axis_discrete and
> wl_pointer.axis with a value of 10
>
> So, the same multiplier of 10 but fast scroll events with more than one click
> per frame appear to get dropped.
>
> kwin (through qtwayland):
> - doesn't handle discrete at all afaict
> - pointer axis events are passed through but I got a bit confused over the
> actual value written on the wire here.
>
> wlroots:
> - pointer axis discrete is sent as-is from libinput
> - pointer axis source is sent as-is from libinput
> - pointer axis value is sent as-is from libinput
>
> Because the physical angle is passed on from libinput as screen coordinates,
> this means wlroots has different scroll behaviour to GNOME/sway and it does
> scroll a larger distance for wheels with higher click angles. This is the
> opposite of the intended behaviour, manufacturers advertise these wheels
> as slow-scrolling wheels. Realistically though the physical movements is the
> same-ish and if a client uses discrete it doesn't matter anyway.
Hmm, that's interesting. We wlroots devs have had assumed you'd want a given
angle to always scroll the same amount of pixels. But this doesn't matter that
much because we have a configuration option for the scroll multiplier anyway.
We still should have good defaults of course.
> xf86-input-libinput:
> - X devices need to set up a scroll distance for scroll motion, the driver
> uses 120 (since Feb, before it was 15). This means any multiple of 120.0
> triggers legacy button emulation in the server, otherwise the number has no
> meaning.
> - for source wheel it uses the discrete value only, so effectively 120 *
> discrete.
>
> This driver had a division by 0 for discrete values of 0, fixed in the
> current release. More on this later.
>
>
>
> Let's look at how wheel events are processed in some of the clients:
>
> GTK:
> - for a discrete event, GDK_SCROLL_UP is passed on flagged as 'emulated'
> - the axis value is passed on as GDK_SCROLL_SMOOTH
>
> Qt:
> - doesn't handle discrete
> - multiplies the value by -12. This gives it a 120-based value which
> matches the Windows API provided the axis value is 10 of course.
> Qt on wlroots is probably off here.
>
> Xwayland:
> - X devices need to set up a scroll distance for scroll motion, Xwayland
> currently uses 1.0 for that. This means any multiple of 1.0 triggers legacy
> button emulation in the server.
> - wl_pointer.axis_discrete are used where available
> - wl_pointer.axis uses the value * 0.1. This effectively means Xwayland
> relies on the deltas to be 10 like in mutter/weston, X clients under
> wlroots will have off wheel click emulation.
Xwayland uses axis_discrete when available, so wlroots should be fine (we sent
the patch doing that when we decided not to implement axis like Weston IIRC).
> With me so far? Hooray, now let's introduce hi-res scrolling.
>
> All the examples below are for a quarter scroll wheel movement, i.e. you get
> 4 hi-res events for every 1 lo-res event. The kernel uses a fraction/multiple
> of 120 for logical clicks, same as the Windows API. So one wheel click is 120,
> half a click is 60, etc. I'm calling this 120-based value the v120 because I'm
> very creative.
>
> kernel:
> - REL_WHEEL for every wheel click
> - REL_WHEEL_HI_RES for every fraction of a wheel click (as portion of 120),
> so a quarter-wheel stop gives you a value of 30.
> - there is no guarantee that REL_WHEEL is sent every full 120 because e.g.
> Logitech mice may reset halfway through a wheel motion. So the two axes
> must be considered completely independent.
>
> libinput:
>
> Basically: the current API is fairly useless in handling fractional scroll
> because libinput uses physical distances. libinput could send four events:
> - value 3.75, discrete 0
> - value 3.75, discrete 0
> - value 3.75, discrete 0
> - value 3.75, discrete 1
>
> Which is technically correct. But callers have no reliable way how much of a
> fraction of a wheel the 3.75 represents until the first discrete event comes
> in. Guessing is unreliable, if you scroll faster you may get 2+ fractional
> units in one event. And mice may reset halfway through a scroll motion so you
> don't always get the same number of events per discrete. So libinput needs some
> extra API that gives us some baseline to compare values against, i.e. same as
> the kernel's 120-based API does.
>
> There are other problems, specifically related to sending discrete 0 events:
>
> weston:
> - Current weston: a discrete value of 0 ends up as value of 0 which weston
> treats as axis_stop event. So our event sequence is axis_source,
> axis_stop, pointer_frame. this breaks scrolling.
>
> mutter:
> - Current mutter: a discrete value of 0 ends up in noops.
> mutter still generates internal clutter events but they are never pushed
> onto the wire. The normal values are ignored, so we basically
> get low-res scrolling just as before.
> Note: this is based on reading the code, not test runs
>
> wlroots:
> - will forward value/discrete as it comes in from libinput
We're in the same boat as Weston here: the special value 0 will be sent as
axis_stop.
>
> kwin:
> - doesn't use discrete so won't be affected by changes
>
> xf86-input-libinput:
> - anything pre 0.28.2 will get a division by 0
> - current state: discrete values of 0 end up as noops and we we get lo-res
> scrolling only
>
> So basically: if we change libinput to send discrete 0 events, we'll break
> weston and all but the most recent xf86-input-libinput. Code changes are needed
> just to cope with the different event sequence, let alone the actual hi-res bits.
Agreed.
> The solution to this is to add a new event, LIBINPUT_EVENT_POINTER_AXIS_WHEEL.
> That event provides libinput_event_pointer_axis_value_v120() which is basically
> a mirror of the kernel API. This event obsoletes POINTER_AXIS events for
> sources WHEEL and WHEEL_TILT, so callers should just ignore those if they
> support the new event. A branch for this is available here:
> https://gitlab.freedesktop.org/whot/libinput/commits/wip/hi-res-scrolling
>
> The event sequence would now be:
>
> - WHEEL: value 3.75, v120 30
> - WHEEL: value 3.75, v120 30
> - WHEEL: value 3.75, v120 30
> - WHEEL: value 3.75, v120 30
> - AXIS: value 3.75, discrete 1, v120 120
>
> Implementation-wise: AXIS and WHEEL are independent, so there's no guarantee
> that they add up to the same 120 values. This new event is easy to add to
> compositors.
This makes sense to me. It's unfortunate to introduce a new API, but I guess as
discussed before there's no way around it.
> Let's look at the Wayland protocol with the current API given hi-res scroll
> events:
> - value 3.75, no discrete event
> - value 3.75, no discrete event
> - value 3.75, no discrete event
> - value 3.75, discrete 1
>
> This is assuming 15 degrees/4 == 3.75 but if the compositor force wheels
> to a scale of 10, the value would be 2.5.
>
> Qt:
> - doesn't use discrete so won't be affected by changes
>
> GTK:
> - generates 4 GDK_SCROLL_SMOOTH events,
> - generates 1 GDK_SCROLL_DOWN event ('emulated')
> - GTK doesn't really care as such about whether the distance-to-wheel is 10 or
> something else, it'll just have different scroll distances (e.g. under
> wlroots).
>
> Afaict this works correctly (tested with stock F29 nautilus)
>
> Xwayland:
> - current Xwayland: values are handled as fraction as before, so we get 3
> smooth-scroll events at 25%. Event 4 has a discrete event which is handled
> as full event, so we scroll by a logical click. Total distance is 175%.
Hmm. Is a logical click 150%? I think I'm missing something here.
But yeah, if we stop sending axis_discrete and continue sending axis, it'll
break clients.
> This means: Xwayland cannot work at all without any extra wayland protocol
> unless it assumes factor 10 is the base unit.
>
>
> So we have at least one ubiquitous Wayland client that cannot handle it.
> And the issues are very similar to the ones libinput's API has, added to that
> is that you cannot guarantee that two events come from the same device.
> What we need is a new event, but I'm struggling to add this to the wayland
> protocol in a good way, mostly for backwards-compatibility reasons.
>
>
> Much of the below would be easier to solve if we can say "if you claim to
> support wl_pointer version 8, you get different behaviour for axis events". I
> don't think we can trust clients to handle this correctly, I got burned by
> those assumptions before (in XI2). So I'm ruling out changing any of the
> existing protocol behaviour.
Hmm. We have to be careful -- anything we decide will be set in stone.
The protocol would look less ugly if we change wl_pointer's meaning in a new
version. (A similar change has already been done to the wl_data_offer API)
Though it's true that until we still have users of the old version we'll need to
be careful.
> My current idea is to add a wl_pointer.axis_v120 event:
>
> wl_pointer.axis_source wheel
> wl_pointer.axis_v120 30
> wl_pointer.frame
> wl_pointer.axis_source wheel
> wl_pointer.axis_v120 30
> wl_pointer.frame
> wl_pointer.axis_source wheel
> wl_pointer.axis_v120 30
> wl_pointer.frame
> wl_pointer.axis_source wheel
> wl_pointer.axis 10
> wl_pointer.axis_discrete 1
> wl_pointer.axis_v120 30
> wl_pointer.frame
I can think of a few issues with this:
* Can the compositor still have a scroll multiplier setting? Is a 120
granularity enough? (ie. what happens if the user has a very precise mouse and
configures a scale multiplier of 0.5?)
Maybe this value could be a wl_fixed and named e.g. axis_fraction?
* What do we do about wl_pointer.axis? Maybe we can just phase it out? I'm not
a fan of having to support two APIs forever. Plus the current axis event is
ill-defined.
* Can't we just define a axis_full_turn event sent after binding to wl_pointer,
which defines what is the amount of axis you need to wait for until you reach
a full turn? (The axis event already carries a wl_fixed value)
> Main problems here: this puts a fair bit of implementation requirements on the
> compositor - skip source/v120/frame for clients < supported version.
Well this is inevitable anyway -- having a new protocol version will require
compositors to add more version checks.
> wl_pointer.source cannot currently occur on its own, it requires a
> wl_pointer.axis event in the current protocol. Same with axis_discrete. Empty
> frames are allowed, I think, but pointless.
>
> It also puts a bit of extra logic in the client, you need to remember if you've
> seen a v120 in this frame and discard the axis/axis_discrete events. If not,
> then axis events must be processed as normal.
>
> A possible workaround would be to add v120 to all axis frames (including
> touchpad scrolling) but that would then put the compositor in charge of
> deciding what distance is a wheel click and encodes that information in the
> protocol.
Side note, the current protocol already requires compositors to decide of an
axis event value for touchpads.
> The v120 event must also carry the equivalent to the wl_pointer.axis value so
> that can be passed on into existing code. So the above would have a v120 event
> with v120 value of 30 and an axis value of 2.5 (if enforcing 10 units per
> wheel click).
>
> Because the hi-res and lo-res sources are independent, we need to allow for a
> 0-value v120 event, which basically just tags that frame as
as low-res?
> The compositor also need to *always* send v120 events for wheel/tilt sources
> because otherwise the client code becomes even more of a nightmare.
> Where not backed by libinput the compositor needs to handle that itself.
Agreed.
> Anyway, the below is the best I've come up with so far though I haven't
> implemented it client-side yet. I'd really appreciate any feedback and/or
> epiphanies.
>
> Cheers,
> Peter
>
> diff --git a/protocol/wayland.xml b/protocol/wayland.xml
> index b1c930d..cd45407 100644
> --- a/protocol/wayland.xml
> +++ b/protocol/wayland.xml
> @@ -1662,7 +1662,7 @@
> </request>
> </interface>
>
> - <interface name="wl_seat" version="7">
> + <interface name="wl_seat" version="8">
> <description summary="group of input devices">
> A seat is a group of keyboards, pointer and touch devices. This
> object is published as a global during start up, or when such a
> @@ -1771,7 +1771,7 @@
>
> </interface>
>
> - <interface name="wl_pointer" version="7">
> + <interface name="wl_pointer" version="8">
> <description summary="pointer input device">
> The wl_pointer interface represents one or more input devices,
> such as mice, which control the pointer location and pointer_focus
> @@ -2092,9 +2092,97 @@
> <arg name="axis" type="uint" enum="axis" summary="axis type"/>
> <arg name="discrete" type="int" summary="number of steps"/>
> </event>
Nit: we generally add a
<!-- Version 8 additions -->
comment here.
> + <event name="axis_v120" since="8">
> + <description summary="axis high-resolution scroll event">
> + Discrete high-resolution scroll information.
> +
> + This event carries high-resolution wheel scroll information,
> + with each multiple of 120 representing one logical scroll step.
> + For example, a v120 of 30 is one-quarter of a logical
> + scroll step in the positive direction, the v120 of -60 is
> + one-half of a logical scroll step in the negative direction.
> + Clients that rely on discrete scrolling should accumulate the
> + v120's to multiples of 120 before processing the event.
> +
> + This event also carries the axis motion value in the same coordinate
> + space as the wl_pointer.axis event's value. See the
> + wl_pointer.axis documentation for details.
Why do we need to duplicate this information?
> + This event is guaranteed for axis movement from axis sources wheel
> + and wheel_tilt even if the underlying device does not support
> + high-resolution scrolling.
> +
> + Where a wl_pointer.axis_source event occurs in the same
> + wl_pointer.frame, the axis source applies to this event.
> +
> + Where wl_pointer.axis and/or wl_pointer.axis_discrete events
> + occur in the same wl_pointer.frame, those events reflect an
> + accumulated scroll amount. Clients handling axis_v120 events can
> + usually ignore the axis and axis_discrete events.
> +
> + For example, a wheel generating events every 25% of each physical
> + wheel detent may create the following event sequence:
> +
> + wl_pointer.axis_source wheel
> + wl_pointer.axis_v120 30
> + wl_pointer.frame
> + wl_pointer.axis_source wheel
> + wl_pointer.axis_v120 30
> + wl_pointer.frame
> + wl_pointer.axis_source wheel
> + wl_pointer.axis_v120 30
> + wl_pointer.frame
> + wl_pointer.axis_source wheel
> + wl_pointer.axis_v120 30
> + wl_pointer.axis 10
> + wl_pointer.axis_discrete 1
> + wl_pointer.frame
> +
> + For backwards compatibility, the wl_pointer.axis event for a wheel
> + or wheel tilt must contain the value of the full wheel movement. It
> + must not be sent for a fraction of a physical detent.
> +
> + For backwards compatibility, the wl_pointer.discrete event for a wheel
> + must contain the value of the full wheel movement. It must not be
> + sent for a fraction of a physical detent and it must not be 0.
> +
> + For backwards compatibility, the wl_pointer.axis_source event in a
> + frame containing only a wl_pointer.axis_v120 event must not be sent
> + to clients that do not support wl_pointer version 8 or later. In
> + other words, the above sequence to such clients is reduced to:
> + wl_pointer.axis_source wheel
> + wl_pointer.axis 10
> + wl_pointer.axis_discrete 1
> + wl_pointer.frame
> +
> + Clients must treat wl_pointer.axis_v120 and
> + wl_pointer.axis/axis_discrete as two independent event sources.
> + There is no guarantee that the steps indicated by the v120 events
> + match the steps by wl_pointer.axis_discrete events. In other words,
> + an accumulated v120 value of 360 does not guarantee an accumulated
> + axis_discrete value of 3.
> +
> + The value of a wl_pointer.axis_v120 event may be 0.
> + This is the case where a low-resolution event has no equivalent
> + high-resolution event. The value of 0 indicates that any
> + wl_pointer.axis/axis_discrete events in the same event can be
> + ignored.
> +
> + The axis motion's coordinate space is identical to the
> + wl_pointer.axis event.
> +
> + The order of wl_pointer.axis_v120 and wl_pointer.axis_source is
> + not guaranteed.
> + </description>
> + <arg name="time" type="uint" summary="timestamp with millisecond granularity"/>
> + <arg name="axis" type="uint" enum="axis" summary="axis type"/>
> + <arg name="motion" type="fixed" summary="length of vector in surface-local coordinate space"/>
> + <arg name="v120" type="int" summary="scroll distance as fraction of 120"/>
> + </event>
> </interface>
>
> - <interface name="wl_keyboard" version="7">
> + <interface name="wl_keyboard" version="8">
> <description summary="keyboard input device">
> The wl_keyboard interface represents one or more keyboards
> associated with a seat.
> @@ -2208,7 +2296,7 @@
> </event>
> </interface>
>
> - <interface name="wl_touch" version="7">
> + <interface name="wl_touch" version="8">
> <description summary="touchscreen input device">
> The wl_touch interface represents a touchscreen
> associated with a seat.
>
> _______________________________________________
> wayland-devel mailing list
> wayland-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/wayland-devel
More information about the wayland-devel
mailing list