[PATCH libinput 0/24] Tablet support

Thu May 22 11:17:22 PDT 2014

On Wed, May 21, 2014 at 10:17 PM, Chandler Paul <thatslyude at gmail.com> wrote:
> Hi! Sorry this took so long to write, I've been spending a lot of my
> time recently trying to understand the libinput code and all of that
> good stuff, and I wanted to make sure I had a decent understanding of it
> before I actually wrote up a response.
>
I'll take a look over the patches and see if I can provide any
comments. I haven't looked at the libinput code before, so I'll
probably be stumbling around for a bit :D

> On Mon, 2014-04-21 at 19:11 +0200, Carlos Garnacho wrote:
>> Hey there!,
>>
>> Here's a few patches to have libinput handle events from tablets,
>> these devices are basically pointer devices, with a varying range
>> of extra buttons (either stylus or "pad" buttons), and extra ABS_*
>> axes. These devices also often offer information about the stylus
>> in use, and its BTN_TOOL_* codes.
>>
>> So I've gone for reusing and extending libinput_event_pointer, adding
>> extra libinput_pointer_axis values, and adding an "axis_frame" event
>> to mark the end of a set of simultaneous axis changes, and a "tool_update"
>> event to mark tool changes (and delimit proximity). These features are
>> only triggered if a new LIBINPUT_DEVICE_CAP_STYLUS capability is set.
> I'm with Peter on splitting these up. It seems kind of inconsistent with
> the rest of libinput (with what I've gathered, anyway). A
> tablet-specific event sounds interesting too, but I feel like
> compressing all of the axis changes into. For now I'm going to work on
> having all the axis changes reported as separate POINTER_AXIS events.
>
>>
>> Caveats:
>>
>> * Many of these devices have also tactile strips or wheels, but these are
>>   unhandled so far... On the devices I've got available for testing, current
>>   kernel support seems varyingly inconsistent:
>>
>>   - One device has 2 strips, which report on ABS_RX/RY (radial??). Min/max
>>     are 0..4096, but the reported values are 1,2,4,8,16... So effectively
>>     a log2 scale, or more graphically a bit shifting over a bunch of 0s,
>>     which is somewhat more resembling to the physical action on the strip.
> Since I'm on a deadline for this and making changes to this in the
> kernel would take too long, I don't think I'm going to advocate for
> changing this behavior right now. Although I do agree that eventually it
> should be changed. Graphically, a bit moving across a field like that
> makes sense, but I think that would be a difficult value to make
> practical use of in an application without changing it to a simple 1-13
> value. If I get far enough that I can start implementing support for
> tactile strips and all those other fancy features some of these tablets
> have I might have it convert the values for tactile strips like that to
> something more usable by default and leave the other axes as-is. I'm
> curious to hear Ping and Jason's opinion on this though, and what kind
> of advantages
>
Having the pad functions work well is very useful, but definitely
secondary to proper pen support. The way our driver (ab)uses the
kernel's event interface to send wheel data is a minor tragedy, and is
something I am very much hoping Benjamin and I will be able to address
in the coming months (I've been side-tracked by other things for the
past few weeks, but just turned my mind back to the problem). If
working around what currently exists is too much of a time sink, I
wouldn't worry too much: pad support is useful, but clearly secondary
to getting the pen working properly.

>>   - Another device has one wheel, reported through ABS_WHEEL. Even though
>>     min/max are reported as [0..1023], on interaction it goes [0..71] (funky
>>     range too)
>>
>>   We could just forward this as-is, but seems hindering enough to do anything
>>   useful with those unless that behavior is corrected.
>>
>>   When supported, IMO it'd make sense to have those axes behave similar to
>>   scroll axes, so the axis value increments or decrements depending on the
>>   direction. I'm not sure if there would be cases where the absolute value
>>   matters here?
>>
>> * One thing worth noting is that axes are currently normalized, to [-1..1]
>>   for stylus tilt, and [0..1] for everything else. I remember Peter's
>>   tablet wayland protocol proposal basically forwarded input_absinfo, this
>>   might not be fully compatible with that, although TBH I'm unsure
>>   clamping/normalization should take place so high in the stack...
> I'm with Peter on this actually. If the axes were used for something
> else I might approve of normalization in libinput but I think having
> absolute values fits more of the use cases for the extra axes found on
> many tablets, especially since Ping said that some of Wacom's in-house
> applications actually need these. I do think however, that maybe we
> should consider clamping axis values with libinput even if we don't
> normalize the axes by default.
>
IMO, normalization makes a lot of sense and libinput is a fine place to do it.

Applications use axis data for various reasons. For instance an app
may read pressure values to adjust brush size, change the ink flow
rate, or possibly even simulate the pressure-varying effects of a
paint knife. When using data, the application can interpret it in two
ways*: as unitless quantity or as a number with some attached unit.
_Many_ applications don't particularly care about units and will
simply ignore any unit information provided. They take the data,
normalize it, and use that fractional value to scale some other
quantity (brush size, flow rate, degree of paint drag). Even data that
you normally associate with a unit is typically treated this way. For
instance, tilt angles are liberally interpreted and as long as the
values are centered and increase/decrease properly most applications
could care less about the underlying units**. For these applications,
normalizing the data up-front just makes things easier: the API is
cleaner since you just move around a bunch of floats, and it saves
applications from doing the normalization that they're going to do
anyway.

Now, that said, there is a small class of applications that make use
of the units for one reason or another. Perhaps it is a drafting
application that measures physical distances in cm between two points,
or perhaps it is that drawing application from earlier now modeling
the physical effects of a given number of newtons of force on the
tool/ink/canvas. These kinds of uses are niche, but we should
absolutely not prevent developers from being able to make them. What
is important to the developer in this case is that the data is
accompanied with enough information to put it into physical context;
at the very least this means transmitting the physical resolution of
the data. Now, you could provide the raw data to an application but
there's not really any point. One application might want to work in
centimeters, the next in inches, the next in meters. In short, though
applications interested in units aren't going to /normalize/ the data,
there's a good chance they're going to /rescale/ the data. And if
they're rescaling the data to their unit-of-choice anyway, it doesn't
really matter what unit the data is ingested in since the resolution
information is available.

Based on that, I think it makes a lot of sense to pre-process the
data. Applications don't use the raw values as-is anyway, so if
pre-processing allows for a simpler and easier-to-understand interface
then it should be seriously considered. Some axes might make more
sense canonicalized rather than normalized (e.g. angles might make
more sense in radians rather than fractions of a [semi-]circle) but
that's a separate issue.

* There is a third vanishingly small class of applications that
neither normalize their data nor care about the unit of the data.
They, for whatever reason, need to know the underlying raw values.
This could be for something like a hardware debug tool or simulator. I
don't think this class of applications is really libinput's target. If
an application really-truly-absolutely needs to know the raw values,
it should use a low-level interface like evdev.

** I'd argue this is the case given that both the Qt and GTK
frameworks have problems with tilt angles. The former specifies data
to be in degrees but doesn't convert non-degree data, while the latter
provides no guarantee of units at all and in practice applications
like GIMP wind up computing the wrong physical angles.

> I've forked libinput and I have a branch where I'm fixing up the patches
> Carlos sent in based on the feedback from Peter. You can find it here:
>
>         https://github.com/Lyude1337/libinput/tree/carlos_cleanup
>
> The history is messy on this, but once this is ready to get sent in as
> actual patches I'll be rebasing the history.
>
> Right now I've removed normalization from my branch, but if someone
> brings up a good reason to actually have libinput handle that then I can
> revert the change.
>
> Another thing I'm considering regarding this is just having libinput
> provide functions/macros for normalizing the values so applications can
> normalize the values easily if they want.
>
That could be an interesting approach as well. After Qt and GTK both
screwed up tilt, I don't trust clients as far as I can throw them.
There should be some way to get normalized/canonical data, whether
that's the default (which I'd currently argue), or through some
conversion functions.

> The way I see it is that we want to allow all of the axes to be used how
> they were meant used. If Wacom isn't normalizing the values in their
> in-house applications I don't think we should either. Again though, I'm
> curious to hear Ping and Jason's take on the reasoning for their tablets
> sending it's data like this.
>
> Cheers,
>         Lyude
>         (a.k.a. Stephen Chandler Paul)
>
>> * No filtering/hysteresis of coords is done yet.
>>
>> Cheers,
>>   Carlos
>>
>> _______________________________________________
>> wayland-devel mailing list
>> wayland-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/wayland-devel
>