Comments about adding tablet support to Wayland

[Dmitry Kazakov]

Hi, all!

I will try to answer all your emails in one post.

1) Tablet vs. Touch. Basically what I am talking about is *not* touch
arbitration and palm recognition. The Wacom driver itself blocks all the
touch events when the stylus is in the proximity, so it performs the
necessary arbitration. What we need is to block several kinds of actions
(read "tools") when the user uses touch and unblock them when the user
works with stylus or mouse. An exact usecase: we need to allow the user to
zoom/pan on the canvas with his fingers, but at the same time real painting
with fingers must be prohibited (it disturbs the painters). Right now it is
almost impossible, because we get no touch events, and instead get
synthesized MousePress/Release/Move events for finger actions, which are
indistinguishable from real mouse events.

2) Buttons vs Tablet Press. Well, to tell you the truth we are not
interested in whether the stylus touched the surface of not. At all. The
stylus has 3 button: the tip itself, and two buttons. These three buttons
can be remapped with the driver (xsetwacom) to any X11 button the user
wants. Therefore, I have absolutely no interest in
TabletTouch/TabletRelease if the only thing they tell me that the stylus
has touched the surface of the tablet with the stylus or not. What I really
need is the TabletButtonPress/Release event, which tells me:

1) Which button (of the three) is pressed
2) Exact position of the stylus
3) Pressure, Tilt, Z-coordinate if available (might be zero/unit for some
of the buttons)

The situation is getting even worse if you look at the feature which
Windows' Wacom driver has (I'm not sure whether this feature is available
in X11 Wacom driver, but it is highly requested by the painters). On
Windows the buttons on the stylus can be switched into "modifier" mode.
That is pressing the button doesn't produce a real button event. You need
to press the  stylus button, and then touch the surface of the tablet with
the tip: only then the app will get mouse button click (right or middle
button usually). If this feature will ever be implemented in X11 Wacom
driver (which is quite desirable), your protocol with TabletTouch/Press
will not work. Please check Wintab protocol docs for more info,
specifically, CSR_SYSBTNMAP attributes [0]

3) Axes resolution. Yes, it is perfectly ok to have a separate function
which tells the physical limits of the axis. What I wanted to say is that
min_value/max_value attributes, which are reported by XInput are not
enough. For rotation I also need to know the mapping of the coordinate
system origin and it's direction (clockwise/counterclockwise).

PS:
Please keep me in CC, I'm having troubles with keeping up with the traffic
in this mailing list.


[0] - http://www.wacomeng.com/windows/docs/Wintab_v140.htm



On Fri, Jun 27, 2014 at 6:53 AM, Jason Gerecke <killertofu at gmail.com> wrote:

> On Wed, Jun 25, 2014 at 5:50 PM, Peter Hutterer
> <peter.hutterer at who-t.net> wrote:
> > Replying to three emails at once here to keep the thread a bit more
> > managable.
> >
> > On Wed, Jun 25, 2014 at 01:38:22PM -0700, Jason Gerecke wrote:
> >> On Wed, Jun 25, 2014 at 12:38 AM, Lyude <thatslyude at gmail.com> wrote:
> >> > On Wed, 2014-06-25 at 11:06 +0400, Dmitry Kazakov wrote:
> >> >> Hi, all!
> >> >>
> >> >>
> >> >> I am a developer from Krita painting application team. We recently
> did
> >> >> quite much work on incorporating better tablet support in Krita. I
> >> >> have several comments about your proposal of the tablet protocol
> >> >> (sorry for nor replying directly, since I wasn't subscribed to the
> >> >> list before).
> >> >
> >> > Benjamin commented on this as I was writing this e-mail, and I
> figured I
> >> > should too: yes, it's awesome to see developers commenting on the
> >> > protocol. A lot of the quirks around this protocol are going to be
> >> > difficult to see without the help of people who have programmed on the
> >> > client side of things as opposed to the compositor side of things. So
> >> > yes, your input is very much appreciated and I thank you for it!
> >> >
> >> >>
> >> >> 1) Axes. There should also be an axis for rotation of the stylus
> >> >> (Artpen) and Tangential Pressure (for the wheel of the Airbrush).
> >> >>
> >> >> 2) There is also Artpen type of stylus. In Qt it is called
> "Rotational
> >> >> Stylus".
> >> >
> >> > Don't worry, we haven't forgotten about these. These will eventually
> be
> >> > added into the protocol. The reason why they're not in this draft is
> >> > because I'm doing this as my Google Summer of Code project, and as
> such
> >> > I'm on a deadline and I have to focus on just getting the basics done
> >> > first before I can focus on all of the other features.
> >
> > Our current approach, both in libinput and the WL protocol should make
> these
> > additions little more than adding a couple of enum, so I think we're good
> > here.
> >
> >> >> 3) Fingers. There is a complication in XInput2 right now, since touch
> >> >> enabled Wacom devices have a special Finger XInput2 device, which
> >> >> provides both interfaces: tablet and touch and therefore generates
> >> >> both types of events. Right now Qt5 still cannot handle it properly,
> >> >> but the work is in progress. From Krita point of view, the main
> >> >> usecase for us is to distinguish whether the user paints with a
> finger
> >> >> of with the stylus. Because most of the users prefer to disable
> >> >> painting with fingers and use it for gestures/UI only (yes, palm
> >> >> detection works with non-100% probability).
> >> >
> >> > Right now libinput handles the finger device as another touchpad,
> since
> >> > that's usually what it is. Your use-case sounds perfectly valid
> though,
> >> > but IMO a better approach would be to add something to the protocol
> for
> >> > touchpads on wayland so that it can be known that they belong to a
> >> > tablet and provide any other sorts of data you might be need, so
> >> > programs like yours can treat them differently.
> >
> > Dmitry, are you talking about pen/touch arbitration, i.e. don't send
> touch
> > events when the pen is in use. If so, that's definitely on the plan, we
> need
> > it for touchpads (disable while typing feature), and we need it for the
> > pen/touch interference.
> >
> > This will be hidden away so you or event the compositor don't have to
> worry
> > about it.
> >
> >> >> 4) Button Press/Release events should come in both cases: when the
> >> >> user clicks on the stylus' buttons and when the stylus touches the
> >> >> surface of the tablet.
> >> >
> >> > I'm not entirely sure that's a good idea. If I'm reading this right,
> you
> >> > mean that additional button presses should be sent when the tool
> touches
> >> > the surface of the tablet. [...]
> >
> > We're already sending out BTN_TOUCH when the tip touches the surface, so
> I
> > think we're good here. Unless Dmitry was referring to something else.
> >
> >
> >> >> 6) It might be a good idea to define the physical properties of the
> >> >> axes. E.g. for tilt, rotation and tangential pressure. Afair, Wacom
> >> >> driver for XInput returns some not-very-obvious values right now. One
> >> >> would need to experiment to know what these numbers mean.
> >> >
> >> > We would all love for this to be the case I promise you, but
> >> > unfortunately it's not that simple for all of the axes. The distance
> >> > axis reports a seemingly meaningless value that can't be converted to
> >> > millimeters very easily. That being said, I have come up with a few
> ways
> >> > that we could actually convert it to millimeters, but this will have
> to
> >> > wait until I've fulfilled the goals for my Google Summer of Code
> Project
> >> > (unless anyone else wants to implement this in the mean time in
> libinput
> >> > of course).
> >> >
> >> > I'll write up the method I've come up with for converting this wild
> >> > value to actual millimeters at some point when I get the chance.
> >> >
> >> While I'm interested in seeing what you've come up with, I would be
> >> very hesitant to integrate the code into libinput. We make *no* claims
> >> about the physical resolution or accuracy of the distance axis. I've
> >> seen the value change by more than 10 units just by switching to a
> >> different pen... There's absolutely nothing stopping us from
> >> introducing a tablet that invalidates any clever code you may come up
> >> with.
> >
> > yeah, I agree with Jason here, let's not pretend we have data we don't
> have.
> > As much as I'd like to attach concrete physical information to all axes
> we
> > can't (well, shouldn't) make it up.
> >
> > Short of keeping a database of each tablet with the offsets and ratios to
> > convert distance values to mm I don't think this is doable, and I'm not a
> > big fan of that either.
> >
> >> > As for the tilt axes of the tools, this is something that could be
> >> > represented in a more meaningful value. We could normalize it to a
> >> > number between 0 and 180°, so you can get the actual tilt in degrees
> as
> >> > opposed to what we currently have. This is something we could pull off
> >> > rather easily, I'll make sure to discuss this with my mentor tomorrow.
> >> >
> >> Alternatively, I would suggest adding an e.g.
> >> `libinput_event_tablet_get_axis_resolution` function. To get a
> >> physical value, a caller would just multiply whatever this function
> >> returns for some axis by the current normalized unitless value for the
> >> same. I suggest this for a few reasons: it is a fairly standard way of
> >> doing things (HID, the kernel, and X all use the same system), is more
> >> efficient (callers working in your preferred unit [e.g. Qt] are spared
> >> doing the conversion but anyone else [e.g. GTK, EFL] will have to do a
> >> second conversion to a different unit), and provides a way of getting
> >> physical information for any axis (if we ever came out with a pen that
> >> accurately measured distance, you wouldn't need to change the
> >> semantics of how distance is reported or sacrifice the now-existant
> >> physical translation information).
> >
> > Jason: how accurate is tilt, and are applications actually using it as
> angle or
> > just as normalized number anyway?
> >
>
> I can't find a mention of the accuracy in any public docs, so I'm a
> little hesitant to give numbers on the list. The spec sheet does show
> it to be accurate to a handful of degrees though.
>
> The second part is a little complicated to answer. Every tilt-enabled
> program I'm aware of uses the data to adjust the brush azimuth. The
> goal is to have the virtual brush angled to be parallel with the
> physical pen at all times. A few applications also calculate an
> altitude angle, deforming the brush from circular to increasingly
> eliptical as the pen becomes more horizontal. Qt and Android (and
> likely EFL based on current discussions) have APIs that specify angles
> in various physical forms: tilt-x/tilt-y in degrees, alt-az in
> radians, etc. GTK on the other hand provides normalized data, but
> neither provides resolution information nor information about what
> "normalized" means. Because of the resulting ambiguity, GIMP and
> Inkscape do their calculations on the assumption that [-1, 1] in GTK
> corresponds to [-180 degrees, +180 degrees] in the physical world. GTK
> actually does its normalization based on the device's min/max though
> (so [-64 degrees, +63 degrees] for our hardware) meaning that GIMP and
> Inkscape wind up calculating incorrect azimuth values. The results
> aren't _that_ wrong though; I'm not aware of anyone having noticed or
> filing a bug about it.
>
> tl;dr, Applications universally /try/ to use the physical angles. Not
> all succeed.
>
> Jason
> ---
> Now instead of four in the eights place /
> you’ve got three, ‘Cause you added one  /
> (That is to say, eight) to the two,     /
> But you can’t take seven from three,    /
> So you look at the sixty-fours....
>
> >> > As for tangential pressure, this is a term I've never actually heard
> of
> >> > before. I don't know what the values from the tablet are supposed to
> >> > correspond to in regards to pressure, so I've added two of our friends
> >> > at Wacom to the CC list help us out (I hope you two don't mind!) on
> >> > this.
> >> > (Jason and Ping, if you guys aren't on the list already, the original
> >> > protocol this e-mail is discussing can be found here:
> >> >
> http://lists.freedesktop.org/archives/wayland-devel/2014-June/015583.html
> >> > )
> >> >
> >> > In regards to rotation and any other axes, I haven't had any contact
> >> > with these yet. So I can't really say much on them.
> >
> > Rotation exists on the mouse/lens cursor tool, but it's a calculation
> based
> > on the tilt x/y axes. For us it'd just be one more axis.
> >
> > Cheers,
> >    Peter
> >
> >> I'm not sure where the term "tangential pressure" came from, but it
> >> can refer either to the fingerwheel on the airbrush tool (expected to
> >> be used to control ink flow rate; value is basically [0, 1]) or the
> >> fingerwheel on the 4D Mouse tool (which is like a spring-loaded
> >> mousewheel that reports how far forward or backward from the neutral
> >> position the wheel is; value is basically [-1, 1]).
> >>
> >> Jason
> >
>



-- 
Dmitry Kazakov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/wayland-devel/attachments/20140630/2824e8da/attachment-0001.html>