Comments about adding tablet support to Wayland

[Ping Cheng]

On Mon, Jun 30, 2014 at 10:36 PM, Dmitry Kazakov <dimula73 at gmail.com> wrote:
>
>> > The situation is getting even worse if you look at the feature which
>> > Windows' Wacom driver has (I'm not sure whether this feature is
>> > available
>> > in X11 Wacom driver, but it is highly requested by the painters). On
>> > Windows the buttons on the stylus can be switched into "modifier" mode.
>> > That is pressing the button doesn't produce a real button event. You
>> > need
>> > to press the  stylus button, and then touch the surface of the tablet
>> > with
>> > the tip: only then the app will get mouse button click (right or middle
>> > button usually). If this feature will ever be implemented in X11 Wacom
>> > driver (which is quite desirable), your protocol with TabletTouch/Press
>> > will not work. Please check Wintab protocol docs for more info,
>> > specifically, CSR_SYSBTNMAP attributes [0]
>>
>> are you talking about Option TPCButton? it does what you describe above,
>> and
>> the earliest reference in the linuxwacom driver I can find for it is 2003
>> :)
>
>
> Probably, yes. How the semantic of TabletTouch/Release will work in that
> case? Technically, a TabletTouch event will have to be sent on each button
> pressed, won't it?

Side switch event (normally translated into right or middle button
click) will only be sent when stylus tip touches tablet. Side switch
event is released if tip stops touching tablet or the switch is
released by end user.

My explanation above is what it works in xf86-input-wacom now. I am
not sure how libinput will handle it.

Ping

>> > 3) Axes resolution. Yes, it is perfectly ok to have a separate function
>> > which tells the physical limits of the axis. What I wanted to say is
>> > that
>> > min_value/max_value attributes, which are reported by XInput are not
>> > enough. For rotation I also need to know the mapping of the coordinate
>> > system origin and it's direction (clockwise/counterclockwise).
>>
>> we're planning to have this well defined for the protocol so that these
>> things can be relied upon.
>>
>> Cheers,
>>    Peter
>>
>> >
>> > PS:
>> > Please keep me in CC, I'm having troubles with keeping up with the
>> > traffic
>> > in this mailing list.
>> >
>> >
>> > [0] - http://www.wacomeng.com/windows/docs/Wintab_v140.htm
>> >
>> >
>> >
>> > On Fri, Jun 27, 2014 at 6:53 AM, Jason Gerecke <killertofu at gmail.com>
>> > wrote:
>> >
>> > > On Wed, Jun 25, 2014 at 5:50 PM, Peter Hutterer
>> > > <peter.hutterer at who-t.net> wrote:
>> > > > Replying to three emails at once here to keep the thread a bit more
>> > > > managable.
>> > > >
>> > > > On Wed, Jun 25, 2014 at 01:38:22PM -0700, Jason Gerecke wrote:
>> > > >> On Wed, Jun 25, 2014 at 12:38 AM, Lyude <thatslyude at gmail.com>
>> > > >> wrote:
>> > > >> > On Wed, 2014-06-25 at 11:06 +0400, Dmitry Kazakov wrote:
>> > > >> >> Hi, all!
>> > > >> >>
>> > > >> >>
>> > > >> >> I am a developer from Krita painting application team. We
>> > > >> >> recently
>> > > did
>> > > >> >> quite much work on incorporating better tablet support in Krita.
>> > > >> >> I
>> > > >> >> have several comments about your proposal of the tablet protocol
>> > > >> >> (sorry for nor replying directly, since I wasn't subscribed to
>> > > >> >> the
>> > > >> >> list before).
>> > > >> >
>> > > >> > Benjamin commented on this as I was writing this e-mail, and I
>> > > figured I
>> > > >> > should too: yes, it's awesome to see developers commenting on the
>> > > >> > protocol. A lot of the quirks around this protocol are going to
>> > > >> > be
>> > > >> > difficult to see without the help of people who have programmed
>> > > >> > on the
>> > > >> > client side of things as opposed to the compositor side of
>> > > >> > things. So
>> > > >> > yes, your input is very much appreciated and I thank you for it!
>> > > >> >
>> > > >> >>
>> > > >> >> 1) Axes. There should also be an axis for rotation of the stylus
>> > > >> >> (Artpen) and Tangential Pressure (for the wheel of the
>> > > >> >> Airbrush).
>> > > >> >>
>> > > >> >> 2) There is also Artpen type of stylus. In Qt it is called
>> > > "Rotational
>> > > >> >> Stylus".
>> > > >> >
>> > > >> > Don't worry, we haven't forgotten about these. These will
>> > > >> > eventually
>> > > be
>> > > >> > added into the protocol. The reason why they're not in this draft
>> > > >> > is
>> > > >> > because I'm doing this as my Google Summer of Code project, and
>> > > >> > as
>> > > such
>> > > >> > I'm on a deadline and I have to focus on just getting the basics
>> > > >> > done
>> > > >> > first before I can focus on all of the other features.
>> > > >
>> > > > Our current approach, both in libinput and the WL protocol should
>> > > > make
>> > > these
>> > > > additions little more than adding a couple of enum, so I think we're
>> > > > good
>> > > > here.
>> > > >
>> > > >> >> 3) Fingers. There is a complication in XInput2 right now, since
>> > > >> >> touch
>> > > >> >> enabled Wacom devices have a special Finger XInput2 device,
>> > > >> >> which
>> > > >> >> provides both interfaces: tablet and touch and therefore
>> > > >> >> generates
>> > > >> >> both types of events. Right now Qt5 still cannot handle it
>> > > >> >> properly,
>> > > >> >> but the work is in progress. From Krita point of view, the main
>> > > >> >> usecase for us is to distinguish whether the user paints with a
>> > > finger
>> > > >> >> of with the stylus. Because most of the users prefer to disable
>> > > >> >> painting with fingers and use it for gestures/UI only (yes, palm
>> > > >> >> detection works with non-100% probability).
>> > > >> >
>> > > >> > Right now libinput handles the finger device as another touchpad,
>> > > since
>> > > >> > that's usually what it is. Your use-case sounds perfectly valid
>> > > though,
>> > > >> > but IMO a better approach would be to add something to the
>> > > >> > protocol
>> > > for
>> > > >> > touchpads on wayland so that it can be known that they belong to
>> > > >> > a
>> > > >> > tablet and provide any other sorts of data you might be need, so
>> > > >> > programs like yours can treat them differently.
>> > > >
>> > > > Dmitry, are you talking about pen/touch arbitration, i.e. don't send
>> > > touch
>> > > > events when the pen is in use. If so, that's definitely on the plan,
>> > > > we
>> > > need
>> > > > it for touchpads (disable while typing feature), and we need it for
>> > > > the
>> > > > pen/touch interference.
>> > > >
>> > > > This will be hidden away so you or event the compositor don't have
>> > > > to
>> > > worry
>> > > > about it.
>> > > >
>> > > >> >> 4) Button Press/Release events should come in both cases: when
>> > > >> >> the
>> > > >> >> user clicks on the stylus' buttons and when the stylus touches
>> > > >> >> the
>> > > >> >> surface of the tablet.
>> > > >> >
>> > > >> > I'm not entirely sure that's a good idea. If I'm reading this
>> > > >> > right,
>> > > you
>> > > >> > mean that additional button presses should be sent when the tool
>> > > touches
>> > > >> > the surface of the tablet. [...]
>> > > >
>> > > > We're already sending out BTN_TOUCH when the tip touches the
>> > > > surface, so
>> > > I
>> > > > think we're good here. Unless Dmitry was referring to something
>> > > > else.
>> > > >
>> > > >
>> > > >> >> 6) It might be a good idea to define the physical properties of
>> > > >> >> the
>> > > >> >> axes. E.g. for tilt, rotation and tangential pressure. Afair,
>> > > >> >> Wacom
>> > > >> >> driver for XInput returns some not-very-obvious values right
>> > > >> >> now. One
>> > > >> >> would need to experiment to know what these numbers mean.
>> > > >> >
>> > > >> > We would all love for this to be the case I promise you, but
>> > > >> > unfortunately it's not that simple for all of the axes. The
>> > > >> > distance
>> > > >> > axis reports a seemingly meaningless value that can't be
>> > > >> > converted to
>> > > >> > millimeters very easily. That being said, I have come up with a
>> > > >> > few
>> > > ways
>> > > >> > that we could actually convert it to millimeters, but this will
>> > > >> > have
>> > > to
>> > > >> > wait until I've fulfilled the goals for my Google Summer of Code
>> > > Project
>> > > >> > (unless anyone else wants to implement this in the mean time in
>> > > libinput
>> > > >> > of course).
>> > > >> >
>> > > >> > I'll write up the method I've come up with for converting this
>> > > >> > wild
>> > > >> > value to actual millimeters at some point when I get the chance.
>> > > >> >
>> > > >> While I'm interested in seeing what you've come up with, I would be
>> > > >> very hesitant to integrate the code into libinput. We make *no*
>> > > >> claims
>> > > >> about the physical resolution or accuracy of the distance axis.
>> > > >> I've
>> > > >> seen the value change by more than 10 units just by switching to a
>> > > >> different pen... There's absolutely nothing stopping us from
>> > > >> introducing a tablet that invalidates any clever code you may come
>> > > >> up
>> > > >> with.
>> > > >
>> > > > yeah, I agree with Jason here, let's not pretend we have data we
>> > > > don't
>> > > have.
>> > > > As much as I'd like to attach concrete physical information to all
>> > > > axes
>> > > we
>> > > > can't (well, shouldn't) make it up.
>> > > >
>> > > > Short of keeping a database of each tablet with the offsets and
>> > > > ratios to
>> > > > convert distance values to mm I don't think this is doable, and I'm
>> > > > not a
>> > > > big fan of that either.
>> > > >
>> > > >> > As for the tilt axes of the tools, this is something that could
>> > > >> > be
>> > > >> > represented in a more meaningful value. We could normalize it to
>> > > >> > a
>> > > >> > number between 0 and 180°, so you can get the actual tilt in
>> > > >> > degrees
>> > > as
>> > > >> > opposed to what we currently have. This is something we could
>> > > >> > pull off
>> > > >> > rather easily, I'll make sure to discuss this with my mentor
>> > > >> > tomorrow.
>> > > >> >
>> > > >> Alternatively, I would suggest adding an e.g.
>> > > >> `libinput_event_tablet_get_axis_resolution` function. To get a
>> > > >> physical value, a caller would just multiply whatever this function
>> > > >> returns for some axis by the current normalized unitless value for
>> > > >> the
>> > > >> same. I suggest this for a few reasons: it is a fairly standard way
>> > > >> of
>> > > >> doing things (HID, the kernel, and X all use the same system), is
>> > > >> more
>> > > >> efficient (callers working in your preferred unit [e.g. Qt] are
>> > > >> spared
>> > > >> doing the conversion but anyone else [e.g. GTK, EFL] will have to
>> > > >> do a
>> > > >> second conversion to a different unit), and provides a way of
>> > > >> getting
>> > > >> physical information for any axis (if we ever came out with a pen
>> > > >> that
>> > > >> accurately measured distance, you wouldn't need to change the
>> > > >> semantics of how distance is reported or sacrifice the now-existant
>> > > >> physical translation information).
>> > > >
>> > > > Jason: how accurate is tilt, and are applications actually using it
>> > > > as
>> > > angle or
>> > > > just as normalized number anyway?
>> > > >
>> > >
>> > > I can't find a mention of the accuracy in any public docs, so I'm a
>> > > little hesitant to give numbers on the list. The spec sheet does show
>> > > it to be accurate to a handful of degrees though.
>> > >
>> > > The second part is a little complicated to answer. Every tilt-enabled
>> > > program I'm aware of uses the data to adjust the brush azimuth. The
>> > > goal is to have the virtual brush angled to be parallel with the
>> > > physical pen at all times. A few applications also calculate an
>> > > altitude angle, deforming the brush from circular to increasingly
>> > > eliptical as the pen becomes more horizontal. Qt and Android (and
>> > > likely EFL based on current discussions) have APIs that specify angles
>> > > in various physical forms: tilt-x/tilt-y in degrees, alt-az in
>> > > radians, etc. GTK on the other hand provides normalized data, but
>> > > neither provides resolution information nor information about what
>> > > "normalized" means. Because of the resulting ambiguity, GIMP and
>> > > Inkscape do their calculations on the assumption that [-1, 1] in GTK
>> > > corresponds to [-180 degrees, +180 degrees] in the physical world. GTK
>> > > actually does its normalization based on the device's min/max though
>> > > (so [-64 degrees, +63 degrees] for our hardware) meaning that GIMP and
>> > > Inkscape wind up calculating incorrect azimuth values. The results
>> > > aren't _that_ wrong though; I'm not aware of anyone having noticed or
>> > > filing a bug about it.
>> > >
>> > > tl;dr, Applications universally /try/ to use the physical angles. Not
>> > > all succeed.
>> > >
>> > > Jason
>> > > ---
>> > > Now instead of four in the eights place /
>> > > you’ve got three, ‘Cause you added one  /
>> > > (That is to say, eight) to the two,     /
>> > > But you can’t take seven from three,    /
>> > > So you look at the sixty-fours....
>> > >
>> > > >> > As for tangential pressure, this is a term I've never actually
>> > > >> > heard
>> > > of
>> > > >> > before. I don't know what the values from the tablet are supposed
>> > > >> > to
>> > > >> > correspond to in regards to pressure, so I've added two of our
>> > > >> > friends
>> > > >> > at Wacom to the CC list help us out (I hope you two don't mind!)
>> > > >> > on
>> > > >> > this.
>> > > >> > (Jason and Ping, if you guys aren't on the list already, the
>> > > >> > original
>> > > >> > protocol this e-mail is discussing can be found here:
>> > > >> >
>> > >
>> > > http://lists.freedesktop.org/archives/wayland-devel/2014-June/015583.html
>> > > >> > )
>> > > >> >
>> > > >> > In regards to rotation and any other axes, I haven't had any
>> > > >> > contact
>> > > >> > with these yet. So I can't really say much on them.
>> > > >
>> > > > Rotation exists on the mouse/lens cursor tool, but it's a
>> > > > calculation
>> > > based
>> > > > on the tilt x/y axes. For us it'd just be one more axis.
>> > > >
>> > > > Cheers,
>> > > >    Peter
>> > > >
>> > > >> I'm not sure where the term "tangential pressure" came from, but it
>> > > >> can refer either to the fingerwheel on the airbrush tool (expected
>> > > >> to
>> > > >> be used to control ink flow rate; value is basically [0, 1]) or the
>> > > >> fingerwheel on the 4D Mouse tool (which is like a spring-loaded
>> > > >> mousewheel that reports how far forward or backward from the
>> > > >> neutral
>> > > >> position the wheel is; value is basically [-1, 1]).
>> > > >>
>> > > >> Jason
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Dmitry Kazakov
>
>
>
>
> --
> Dmitry Kazakov