Comments about adding tablet support to Wayland

[Jason Gerecke]

On Wed, Jun 25, 2014 at 12:38 AM, Lyude <thatslyude at gmail.com> wrote:
> On Wed, 2014-06-25 at 11:06 +0400, Dmitry Kazakov wrote:
>> Hi, all!
>>
>>
>> I am a developer from Krita painting application team. We recently did
>> quite much work on incorporating better tablet support in Krita. I
>> have several comments about your proposal of the tablet protocol
>> (sorry for nor replying directly, since I wasn't subscribed to the
>> list before).
>
> Benjamin commented on this as I was writing this e-mail, and I figured I
> should too: yes, it's awesome to see developers commenting on the
> protocol. A lot of the quirks around this protocol are going to be
> difficult to see without the help of people who have programmed on the
> client side of things as opposed to the compositor side of things. So
> yes, your input is very much appreciated and I thank you for it!
>
>>
>> 1) Axes. There should also be an axis for rotation of the stylus
>> (Artpen) and Tangential Pressure (for the wheel of the Airbrush).
>>
>> 2) There is also Artpen type of stylus. In Qt it is called "Rotational
>> Stylus".
>
> Don't worry, we haven't forgotten about these. These will eventually be
> added into the protocol. The reason why they're not in this draft is
> because I'm doing this as my Google Summer of Code project, and as such
> I'm on a deadline and I have to focus on just getting the basics done
> first before I can focus on all of the other features.
>
>>
>> 3) Fingers. There is a complication in XInput2 right now, since touch
>> enabled Wacom devices have a special Finger XInput2 device, which
>> provides both interfaces: tablet and touch and therefore generates
>> both types of events. Right now Qt5 still cannot handle it properly,
>> but the work is in progress. From Krita point of view, the main
>> usecase for us is to distinguish whether the user paints with a finger
>> of with the stylus. Because most of the users prefer to disable
>> painting with fingers and use it for gestures/UI only (yes, palm
>> detection works with non-100% probability).
>
> Right now libinput handles the finger device as another touchpad, since
> that's usually what it is. Your use-case sounds perfectly valid though,
> but IMO a better approach would be to add something to the protocol for
> touchpads on wayland so that it can be known that they belong to a
> tablet and provide any other sorts of data you might be need, so
> programs like yours can treat them differently.
>
>>
>> 4) Button Press/Release events should come in both cases: when the
>> user clicks on the stylus' buttons and when the stylus touches the
>> surface of the tablet.
>
> I'm not entirely sure that's a good idea. If I'm reading this right, you
> mean that additional button presses should be sent when the tool touches
> the surface of the tablet. This could be done, but this would result in
> us either having to:
> - Send release events for all the currently held down buttons when the
> stylus makes contact with the surface of the tablet, then immediately
> send press events for all of the buttons or
> - Just send redundant press events when the tool makes contact with the
> surface
>
> The issue with this is that it's very misleading to a wayland client as
> to when the buttons are actually being pressed or not, and it doesn't
> provide much in the way of extra useful information for the majority of
> use cases.
>
>>
>> 6) It might be a good idea to define the physical properties of the
>> axes. E.g. for tilt, rotation and tangential pressure. Afair, Wacom
>> driver for XInput returns some not-very-obvious values right now. One
>> would need to experiment to know what these numbers mean.
>
> We would all love for this to be the case I promise you, but
> unfortunately it's not that simple for all of the axes. The distance
> axis reports a seemingly meaningless value that can't be converted to
> millimeters very easily. That being said, I have come up with a few ways
> that we could actually convert it to millimeters, but this will have to
> wait until I've fulfilled the goals for my Google Summer of Code Project
> (unless anyone else wants to implement this in the mean time in libinput
> of course).
>
> I'll write up the method I've come up with for converting this wild
> value to actual millimeters at some point when I get the chance.
>
While I'm interested in seeing what you've come up with, I would be
very hesitant to integrate the code into libinput. We make *no* claims
about the physical resolution or accuracy of the distance axis. I've
seen the value change by more than 10 units just by switching to a
different pen... There's absolutely nothing stopping us from
introducing a tablet that invalidates any clever code you may come up
with.

> As for the tilt axes of the tools, this is something that could be
> represented in a more meaningful value. We could normalize it to a
> number between 0 and 180°, so you can get the actual tilt in degrees as
> opposed to what we currently have. This is something we could pull off
> rather easily, I'll make sure to discuss this with my mentor tomorrow.
>
Alternatively, I would suggest adding an e.g.
`libinput_event_tablet_get_axis_resolution` function. To get a
physical value, a caller would just multiply whatever this function
returns for some axis by the current normalized unitless value for the
same. I suggest this for a few reasons: it is a fairly standard way of
doing things (HID, the kernel, and X all use the same system), is more
efficient (callers working in your preferred unit [e.g. Qt] are spared
doing the conversion but anyone else [e.g. GTK, EFL] will have to do a
second conversion to a different unit), and provides a way of getting
physical information for any axis (if we ever came out with a pen that
accurately measured distance, you wouldn't need to change the
semantics of how distance is reported or sacrifice the now-existant
physical translation information).

> As for tangential pressure, this is a term I've never actually heard of
> before. I don't know what the values from the tablet are supposed to
> correspond to in regards to pressure, so I've added two of our friends
> at Wacom to the CC list help us out (I hope you two don't mind!) on
> this.
> (Jason and Ping, if you guys aren't on the list already, the original
> protocol this e-mail is discussing can be found here:
> http://lists.freedesktop.org/archives/wayland-devel/2014-June/015583.html
> )
>
> In regards to rotation and any other axes, I haven't had any contact
> with these yet. So I can't really say much on them.
>
> Cheers,
>         Lyude

I'm not sure where the term "tangential pressure" came from, but it
can refer either to the fingerwheel on the airbrush tool (expected to
be used to control ink flow rate; value is basically [0, 1]) or the
fingerwheel on the 4D Mouse tool (which is like a spring-loaded
mousewheel that reports how far forward or backward from the neutral
position the wheel is; value is basically [-1, 1]).

Jason
---
Now instead of four in the eights place /
you’ve got three, ‘Cause you added one  /
(That is to say, eight) to the two,     /
But you can’t take seven from three,    /
So you look at the sixty-fours....