[RFC] libinputmapper: Input device configuration for graphic-servers

[David Herrmann]

Hi Peter

On Mon, May 27, 2013 at 12:35 PM, Peter Hutterer
<peter.hutterer at who-t.net> wrote:
> On Fri, May 24, 2013 at 05:07:14PM +0200, David Herrmann wrote:
> [...]
>> >> This library is intended to solve the classification/detection
>> >> problem. While the kernel evdev-interface provides us a bunch of
>> >> information for each device, it doesn't provide any classification of
>> >> the device (mostly because it would be unreliable). So libinputmapper
>> >> as I see it is supposed to replace the heuristics that we currently
>> >> have (KEY_A-KEY_Z => keyboard, REL_X/Y => mouse, ...) with a more
>> >> user-controlled interface.
>> >>
>> >> So a *compositor* defines the type of devices, that it wants to
>> >> support. As an example, it needs mouse+touchpad devices as
>> >> mouse-input, keyboards as keyboard input. It then listens for all
>> >> input hotplug events via udev, calls libinputmapper to provide tags
>> >> for the devices and then checks, whether it's one of the devices that
>> >> it wants. Client-side support is not involved, yet.
>> >
>> > ok, that makes sense to me and is a good idea. it's a combined version of
>> > (formerly) udev's input_id tool and the Match* system that X has, in the
>> > form of a library with simple adjustment based on textfiles.
>> >
>> > good. and sorry, I did somewhat tie all this in with the client side as
>> > well, which is where the confusion came from.
>> >
>> > I think the tagging library will be quite private to the compositor
>> > implementation (even if that means all compositors). in the X case, we have
>> > evdev and the wacom driver for tablet devices since they're different
>> > enough to basic tablets. I suspect something like that
>> > may be required for wayland sooner or later.
>>
>> input_id+Match* seems to be a good description. That's what I had in
>> mind. But I don't see a reason to keep it private to the compositor.
>> The compositor is the #1 user of the library, but whatever API is used
>> to forward events to a client, we shouldn't hide this source. We can
>> save all other kinds of useful information in these files. A client
>> could retrieve device-geometry information or other static data.
>> I designed the configuration to allow a global database with local
>> additions. So a compositor can have a different configuration than a
>> client. But both have the same global base/fallback configuration.
>
> ok. the reason I assumed private is because you need to decide where the
> level of abstraction sits and I think we don't think of it the same way
> there.
>
> you can tag a device as tablet, but that doesn't tell you which driver to
> use in the compositor to parse the data (see wacom example). so now the
> compositor has another lookup table so it knows which module to load for
> this device. this lookup table can either be duplicated across compositors,
> or be moved into a library. and now you have your libinputmodulemapper on
> top (well, on the side) of libinputmapper.

I don't understand. What's wrong with linking device-TAGs to drivers
in the compositor? So the compositor loads all drivers dynamically and
each driver specifies which tags it supports. A new device is then
assigned to the driver that currently is associated with the tag.

If you want to overwrite that, add a "[driver]" section to the
device-config/rule and use "bind = <some-driver-name>". The compositor
then needs to support the "[driver]" section and can see whether it
should force-bind another driver than the default.

Or even better, if you have multiple tags for tablets, like TAG_TABLET
and TAG_WACOM_TABLET, you can define both on wacom tablets but only
TAG_TABLET on other random tablets. Then bind the normal
tablet/mouse/whatever driver to TAG_TABLET and your wacom driver to
TAG_WACOM_TABLET. Of course, TAG_WACOM_TABLET then needs a higher
priority than TAG_TABLET.
This will work whether the wacom-driver is loaded or not.

> likewise, as in another part of this thread: the compositor may not care
> about the distinction between joystick/gamepad but the client may. so again
> we have two different use-cases and mapping requirements here.

Then the compositor just binds it's gaming-device-driver to
(TAG_JOYSTICK | TAG_GAMEPAD). A client may use TAG_JOYSTICK or
TAG_GAMEPAD separately. I don't see the problem here?

>> > so the tagging library would have to tag a device not just as graphics
>> > tablet but more as input module selection library. which again, means it's a
>> > library version of input_id and Match*, etc.
>> >
>> > what you will also need then is for drivers to provide hooks to notify what
>> > they can deal with. i.e. libtouchpadcommon needs to install data files for
>> > libinputmapper to notify that it's now taking over touchpads.
>>
>> I wanted to keep this private to the compositor. So the compositor has
>> a list of drivers and a libinputmapper context. It then retrieves the
>> tags for a device and assigns the device depending on the tags to the
>> right driver. So libinputmapper always returns the same information.
>> It's up to the compositor to use this according to it's capabilities.
>> Or what would be the benefit of telling libinputmapper which drivers
>> are available?
>>
>> I also don't understand what you mean by "install data files". What
>> data-files would libtouchpadcommon install?
>
> once you have input modules that are externalised (which they should be to
> allow for re-use across compositors) you'll have to deal with the issue of
> _which_ modules are actually installed. so you run into the same situation
> as we do with X input modules: evdev handles anything, but synaptics will
> install a MatchIsTouchpad snippet to take over all touchpads.
>
> on top of that you have per-device quirks. the X wacom driver knows it can
> handle some n-trig and waltop devices, but it can't handle all tablets. so
> it installs MatchProduct snippets for those devices.
>
> you could handle all that in the compositor, but it can get complex once you
> look at more than 3+ modules and several compositors that would duplicate
> this.

That is exactly why I associated TAGs with a protocol. Lets stay with
the example TAG_GAMEPAD and TAG_JOYSTICK. Both tags specify exactly
what data is reported by the kernel but leave *additional* room for
device specific behavior. A compositor can specify a
gaming-device-driver and binds it to (TAG_GAMEPAD | TAG_JOYSTICK),
because it knows that this generic driver supports both protocols.

A specific gamepad driver (on client or compositor side) that can deal
with the TAG_GAMEPAD protocol, but not with TAG_JOYSTICK, can bind to
TAG_GAMEPAD with a higher priority. So the compositor binds the
gamepad driver to TAG_GAMEPAD devices if it is loaded. If not, it
falls back to the generic gaming-device-driver.

But note that the gamepad driver does _not_ need a MatchProduct
helper. It doesn't make sense in this library. Because the TAG_GAMEPAD
tag was created exactly for the needs of the gamepad driver. So the
MatchProduct is part of libinputmapper and only compatible devices get
this TAG_GAMEPAD tag.

If you create a new wacom driver that deals only with specific wacom
tablets, you simply define a new TAG. Something like TAG_WACOM_GEN7 or
something like this. And only devices that support the
wacom-generation-7 features get this tag.

One might wonder whether this will produce a vast amount of tags
pretty soon, but that's acceptable, imo. Sure, we need to change the
bitflags in the API to integers to allow (nearly) arbitrarily huge
number of tags. But that's also why I required a compositor/client to
pass supported tags to the context initialization. This allows the
library to apply only heuristics for supported tags and ignore all
unused tags.
And it allows to drop all match-rules for unneeded tags. Or other
crazy optimizations.

>> My idea was to keep config-files generic. One file with "match"-rules
>> and a bunch of "device"-rules files. The match-rules would be used by
>> libinputmapper to find the device-rule (if any) files that should be
>> "applied" to the device. The device-rules contain different sections.
>> libinputmapper only parses one section which affects tagging. This is
>> optional and only needed if the default tagging-heuristics are wrong
>> for that device.
>> All other information in the device-rules files are ignored, but we
>> can allow the compositor to retrieve the file-name. It can then pass
>> the file-name to libtouchpadcommon which can then look whether the
>> user specified a [libtouchpadcommon] section for the device. If not,
>> it uses the default values.
>
> question here:
> do you plan for multiple rules files per device? main reason here is that
> it'd be unwieldly to have [foobar] sections in a rules file when the foobar
> module isn't actually installed. and as said above, a driver may actually
> need to provide extra rules files on top of libinputmapper.

Honestly, I don't know. Another approach would be to install rules by
device, not by driver. So you add your wacom-tablet-v7.rules with the
wacom-tablet-v7 package.
Or we use rule-directories instead of rule-files, so each package can
install additional rules into the different device-specific
directories.
As you might see, I am open for suggestions.

>> configuration example:
>> Four files, a "match"-file and 3 device-rules. It assigns
>> "wiimote.rule" to all Wii-Remote input devices so they are ignored
>> (which is what we do for Xorg, too). And it provides a custom rule for
>> a specific device which can be installed by the user. It doesn't
>> overwrite tagging-rules, so the defaults are applied. But it provides
>> additional information for xkbcommon to set the layout different for
>> this device.
>> The third file is matched on every device and causes an "en" layout
>> for xkbcommon to be applied.
>> (The match-logic can be extended, this is just an example that assumes
>> we stop on the first match..)
>>
>> file: inputmapper.match:
>> [match]
>> name = Nintendo Wii Remote*
>> rule = gamepad/wiimote.rule
>> [match]
>> vid = 0x3663
>> pid = 0x0010
>> rule = misc/custom.rule
>> [match]
>> rule = default.rule
>>
>> file: gamepad/wiimote.rule
>> [tags]
>> # clear tags to avoid using Wii-Remote IR/Accel-data erroneously as mouse-input
>> set =
>>
>> file: misc/custom.rule
>> [xkbcommon]
>> layout = de
>> variant = nodeadkeys
>>
>> file: default.rule
>> [xkbcommon]
>> layout = en
>>
>> >> While device-classification might seem simple, there is a reason why
>> >> you guys wrote the highly customizable X11 InputDevice configuration
>> >> infrastructure. We cannot be sure that we do it right! And we should
>> >> never claim that. So libinputmapper comes with a configuration-parser
>> >> that allows users to tag devices manually. Example: If I have an
>> >> external Numpad with KEY_0 to KEY_9, but it's not correctly detected
>> >> as such, I could easily tag this device as KEYBOARD in a config-file
>> >> and my compositor would pick it up because libinputmapper now marks it
>> >> as KEYBOARD.
>> >>
>> >> This removes the need of device-specific quirks in the
>> >> driver-libraries itself. But if we allow users to re-tag custom
>> >> devices, we must also guarantee that the driver understands the
>> >> device-input. That's why I came up with the mapping-tables. So we
>> >> _define_ how each device must behave to be tagged as such and the
>> >> driver-library can then simply expect every device to comply with
>> >> these rules (ABS_X/Y for TOUCHPAD devices means touch-position, for
>> >> example, not accelerometer data or anything else). But to avoid being
>> >> too restrictive, we allow simple 1-to-1 remappings for devices, that
>> >> report the same data but with different codes (ABS_RX/RY instead of
>> >> ABS_X/Y in the TOUCHPAD example).
>> >
>> > while there is quite a bit of overlap, I'm not convinced yet that a mapping
>> > library and a tagging library are the same thing - or should be combined.
>> > the two issues are imo orthogonal and though both would use a device
>> > database, one is tied more to the compositor's needs (i.e. which driver) and
>> > one to truly fix up kernel issues (could thus be useful for X too).
>> > this doesn't change the flow of what you describe in the next paragraph
>> > below, merely whether it's one or two components involved.
>>
>> Yes, you are right. The mapping makes more sense as part of libevdev_read.
>> And maybe libinputmapper should rather be named something like
>> "device-database", libdevdb? Because what it does is basically provide
>> additional information per device that the kernel cannot provide.
>
> yes, but I'd consider the naming a side issue until it's clear what exactly
> we're trying to achive :) once that is clear, the name may just be obvious
> anyway.

I am very aware that you have a lot more experience in that field and
if my suggestions are too complex, weird, ... just tell me. But my
main goal is to avoid hardcoding device detection in drivers. Because
this requires each driver to provide user-configuration to overwrite
these heuristics. And we don't have a central place like the xserver,
anymore.
So I try to provide a central device-configuration library that is
independent of the compositor/client.

Thanks for the feedback!
Cheers
David