Comments about adding tablet support to Wayland

[Peter Hutterer]

Replying to three emails at once here to keep the thread a bit more
managable.

On Wed, Jun 25, 2014 at 01:38:22PM -0700, Jason Gerecke wrote:
> On Wed, Jun 25, 2014 at 12:38 AM, Lyude <thatslyude at gmail.com> wrote:
> > On Wed, 2014-06-25 at 11:06 +0400, Dmitry Kazakov wrote:
> >> Hi, all!
> >>
> >>
> >> I am a developer from Krita painting application team. We recently did
> >> quite much work on incorporating better tablet support in Krita. I
> >> have several comments about your proposal of the tablet protocol
> >> (sorry for nor replying directly, since I wasn't subscribed to the
> >> list before).
> >
> > Benjamin commented on this as I was writing this e-mail, and I figured I
> > should too: yes, it's awesome to see developers commenting on the
> > protocol. A lot of the quirks around this protocol are going to be
> > difficult to see without the help of people who have programmed on the
> > client side of things as opposed to the compositor side of things. So
> > yes, your input is very much appreciated and I thank you for it!
> >
> >>
> >> 1) Axes. There should also be an axis for rotation of the stylus
> >> (Artpen) and Tangential Pressure (for the wheel of the Airbrush).
> >>
> >> 2) There is also Artpen type of stylus. In Qt it is called "Rotational
> >> Stylus".
> >
> > Don't worry, we haven't forgotten about these. These will eventually be
> > added into the protocol. The reason why they're not in this draft is
> > because I'm doing this as my Google Summer of Code project, and as such
> > I'm on a deadline and I have to focus on just getting the basics done
> > first before I can focus on all of the other features.

Our current approach, both in libinput and the WL protocol should make these
additions little more than adding a couple of enum, so I think we're good
here.

> >> 3) Fingers. There is a complication in XInput2 right now, since touch
> >> enabled Wacom devices have a special Finger XInput2 device, which
> >> provides both interfaces: tablet and touch and therefore generates
> >> both types of events. Right now Qt5 still cannot handle it properly,
> >> but the work is in progress. From Krita point of view, the main
> >> usecase for us is to distinguish whether the user paints with a finger
> >> of with the stylus. Because most of the users prefer to disable
> >> painting with fingers and use it for gestures/UI only (yes, palm
> >> detection works with non-100% probability).
> >
> > Right now libinput handles the finger device as another touchpad, since
> > that's usually what it is. Your use-case sounds perfectly valid though,
> > but IMO a better approach would be to add something to the protocol for
> > touchpads on wayland so that it can be known that they belong to a
> > tablet and provide any other sorts of data you might be need, so
> > programs like yours can treat them differently.

Dmitry, are you talking about pen/touch arbitration, i.e. don't send touch
events when the pen is in use. If so, that's definitely on the plan, we need
it for touchpads (disable while typing feature), and we need it for the
pen/touch interference.

This will be hidden away so you or event the compositor don't have to worry
about it.

> >> 4) Button Press/Release events should come in both cases: when the
> >> user clicks on the stylus' buttons and when the stylus touches the
> >> surface of the tablet.
> >
> > I'm not entirely sure that's a good idea. If I'm reading this right, you
> > mean that additional button presses should be sent when the tool touches
> > the surface of the tablet. [...]

We're already sending out BTN_TOUCH when the tip touches the surface, so I
think we're good here. Unless Dmitry was referring to something else.


> >> 6) It might be a good idea to define the physical properties of the
> >> axes. E.g. for tilt, rotation and tangential pressure. Afair, Wacom
> >> driver for XInput returns some not-very-obvious values right now. One
> >> would need to experiment to know what these numbers mean.
> >
> > We would all love for this to be the case I promise you, but
> > unfortunately it's not that simple for all of the axes. The distance
> > axis reports a seemingly meaningless value that can't be converted to
> > millimeters very easily. That being said, I have come up with a few ways
> > that we could actually convert it to millimeters, but this will have to
> > wait until I've fulfilled the goals for my Google Summer of Code Project
> > (unless anyone else wants to implement this in the mean time in libinput
> > of course).
> >
> > I'll write up the method I've come up with for converting this wild
> > value to actual millimeters at some point when I get the chance.
> >
> While I'm interested in seeing what you've come up with, I would be
> very hesitant to integrate the code into libinput. We make *no* claims
> about the physical resolution or accuracy of the distance axis. I've
> seen the value change by more than 10 units just by switching to a
> different pen... There's absolutely nothing stopping us from
> introducing a tablet that invalidates any clever code you may come up
> with.

yeah, I agree with Jason here, let's not pretend we have data we don't have.
As much as I'd like to attach concrete physical information to all axes we
can't (well, shouldn't) make it up.

Short of keeping a database of each tablet with the offsets and ratios to
convert distance values to mm I don't think this is doable, and I'm not a
big fan of that either.

> > As for the tilt axes of the tools, this is something that could be
> > represented in a more meaningful value. We could normalize it to a
> > number between 0 and 180°, so you can get the actual tilt in degrees as
> > opposed to what we currently have. This is something we could pull off
> > rather easily, I'll make sure to discuss this with my mentor tomorrow.
> >
> Alternatively, I would suggest adding an e.g.
> `libinput_event_tablet_get_axis_resolution` function. To get a
> physical value, a caller would just multiply whatever this function
> returns for some axis by the current normalized unitless value for the
> same. I suggest this for a few reasons: it is a fairly standard way of
> doing things (HID, the kernel, and X all use the same system), is more
> efficient (callers working in your preferred unit [e.g. Qt] are spared
> doing the conversion but anyone else [e.g. GTK, EFL] will have to do a
> second conversion to a different unit), and provides a way of getting
> physical information for any axis (if we ever came out with a pen that
> accurately measured distance, you wouldn't need to change the
> semantics of how distance is reported or sacrifice the now-existant
> physical translation information).

Jason: how accurate is tilt, and are applications actually using it as angle or
just as normalized number anyway?

> > As for tangential pressure, this is a term I've never actually heard of
> > before. I don't know what the values from the tablet are supposed to
> > correspond to in regards to pressure, so I've added two of our friends
> > at Wacom to the CC list help us out (I hope you two don't mind!) on
> > this.
> > (Jason and Ping, if you guys aren't on the list already, the original
> > protocol this e-mail is discussing can be found here:
> > http://lists.freedesktop.org/archives/wayland-devel/2014-June/015583.html
> > )
> >
> > In regards to rotation and any other axes, I haven't had any contact
> > with these yet. So I can't really say much on them.

Rotation exists on the mouse/lens cursor tool, but it's a calculation based
on the tilt x/y axes. For us it'd just be one more axis.

Cheers,
   Peter

> I'm not sure where the term "tangential pressure" came from, but it
> can refer either to the fingerwheel on the airbrush tool (expected to
> be used to control ink flow rate; value is basically [0, 1]) or the
> fingerwheel on the 4D Mouse tool (which is like a spring-loaded
> mousewheel that reports how far forward or backward from the neutral
> position the wheel is; value is basically [-1, 1]).
> 
> Jason