[RFC] libinputmapper: Input device configuration for graphic-servers

[David Herrmann]

Hi Peter

On Thu, May 16, 2013 at 7:37 AM, Peter Hutterer
<peter.hutterer at who-t.net> wrote:
> On Sun, May 12, 2013 at 04:20:59PM +0200, David Herrmann wrote:
[..]
>> So what is the proposed solution?
>> My recommendation is, that compositors still search for devices via
>> udev and use device drivers like libxkbcommon. So linux evdev handling
>> is still controlled by the compositor. However, I'd like to see
>> something like my libinputmapper proposal being used for device
>> detection and classification.
>>
>> libinputmapper provides an "inmap_evdev" object which reads device
>> information from an evdev-fd or sysfs /sys/class/input/input<num>
>> path, performs some heuristics to classify it and searches it's global
>> database for known fixups for broken devices.
>> It then provides "capabilities" to the caller, which allow them to see
>> what drivers to load on the device. And it provides a very simple
>> mapping table that allows to apply fixup mappings for broken devices.
>> These mappings are simple 1-to-1 mappings that are supposed to be
>> applied before drivers handle the input. This is to avoid
>> device-specific fixup in the drivers and move all this to the
>> inputmapper. An example would be a remapping for gamepads that report
>> BTN_A instead of BTN_NORTH, but we cannot fix them in the kernel for
>> backwards-compatibility reasons. The gamepad-driver can then assume
>> that if it receives BTN_NORTH, it is guaranteed to be BTN_NORTH and
>> doesn't need to special case xbox360/etc. controllers, because they're
>> broken.
>
> I think evdev is exactly that interface and apparently it doesn't work.
>
> if you want a mapping table, you need a per-client table because sooner or
> later you have a client that needs BTN_FOO when the kernel gives you BTN_BAR
> and you can't change the client to fix it.
>
> i.e. the same issue evdev has now, having a global remapping table just
> moves the problem down by 2 years.
>
> a mapping table is good, but you probably want two stages of mapping: one
> that's used in the compositor for truly broken devices that for some reason
> can't be fixed in the kernel, and one that's used on a per-client basis. and
> you'll likely want to be able to overide the client-specific from outside
> the client too.

IMHO, the problem with evdev is, that it doesn't provide device
classes. The only class we have is "this is an input device". All
other event-flags can be combined in whatever way we want.

So like 10 years ago when the first gamepad driver was introduced, we
added some mapping that was unique to this device (the device was
probably unique, too). Some time later, we added some other
gamepad-like driver with a different mapping (as it was probably a
very different device-type, back then, and we didn't see it coming
that this will become a wide-spread device-type).
However, today we notice that a "GamePad" is an established type of
device (like a touchpad), but we have tons of different mappings in
the kernel for backwards-compatibility reasons. I can see that this
kind of development can happen again (and very likely it _will_ happen
again) and it will happen for all kinds of devices.

But that's why I designed the proposal from a compositor's view
instead of from a kernel's view.

A touchpad driver of the compositor needs to know exactly what kind of
events it gets from the kernel. If it gets wrong events, it will
misbehave. As we cannot guarantee that all kernel drivers behave the
same way, the compositor's touchpad driver needs to work around all
these little details on a per-device basis.
To avoid this, I tried to abstract the touchpad-protocol and moved
per-device handling into a separate library. It detects all devices
that can serve as a touchpad and fixes trivial (1-to-1 mapping)
incompatibilities. This removes all per-device handling from the
touchpad driver and it can expect all input it gets to be conform with
a "touchpad" protocol.
And in fact, it removes this from all the compositor's input drivers.
So I think of it more like a "lib-detect-and-make-compat".

All devices that do not fall into one of the categories (I called it
capability), will be handled as custom devices. So if we want an input
driver for a new fancy device, then we need a custom driver, anyway
(or adjust a generic driver to handle both). If at some point it turns
out, that this kind of device becomes more established, we can add a
new capability for it. Or we try extending an existing capability in a
backwards-compatible way. We can then remove the "custom-device
handling" from the input-driver and instead extend/write a generic
driver for the new capability.


So I cannot follow how you think this will have the same problems as
evdev? Or, let's ask the inverse question: How does this differ from
the X11 model where we move the "custom device handling" into the
drivers?

>> libinputmapper would use some static heuristics for all this, but
>> additionally parse user-configuration. A configuration file contains
>> "[match]" entries, which specify device-configurations to load on
>> mtached devices. The device-configurations then contain different
>> blocks which can overwrite the auto-detected capabilities or provide
>> fixup-mappings.
>> But other configuration-sections can also be provided in the same
>> file. For instance an [xkb] group could be specified to set XKB
>> layout/variant/options. This won't be parsed by libinputmapper, but it
>> would provide the matching and forward the device-configuration to the
>> caller, which can pass it on to the driver.
>> Additionally, API users can provide separate (local)
>> configuration-files that allow users to change the configuration for
>> kmscon one way and another way for gnome-shell (or similar).
>>
>> One can think of these configurations to replace the old
>> /etc/X11/xorg.conf.d/*.conf "InputClass" configuration items. Except
>> that the "matching" is offloaded to a separate file, and the "to be
>> applied configurations" are specified in "rule" files, that are linked
>> to from the "match" configurations. See below for syntax-examples.
>>
>>
>> I haven't implemented the library, yet. I am working on this. However,
>> I wrote up an API proposal that I attached inline below. kmscon input
>> handling is what I need it for, so I want something like this, anyway.
>> But I thought other compositors might be interested in this, too.
>>
>> Comments are welcome!
>> Regards
>> David
>>
>>
>
> [...]
>
>> /**
>>  * @enum inmap_capability
>>  * Generic device capabilities
>>  *
>>  * Device capabilities describe what generic interface is provided by the
>>  * detected device. A device can support any combination of them (or none).
>>  * Each interface follows specific rules. If a device does not follow these
>>  * rules, we must provide a mapping table so the correct data is reported.
>>  *
>>  * Capabilities can be added for all kinds of generic devices. However, custom
>>  * devices with model-specific features do not belong here, as their drivers
>>  * need to perform device-detection themselves, anyway.
>>  *
>>  * Furthermore, each capability needs a well-defined protocol. They are mostly
>>  * already provided by the kernel. But there are many devices and drivers
>>  * which provide inconsistent behavior. That's why each capability has to
>>  * be consistent and coherent and misbehaving devices get fixup mappings, if
>>  * we cannot fix it in the kernel device driver itself.
>>  */
>> enum inmap_capability {
>> /**
>> * @INMAP_CAP_KEYBOARD
>> * Keyboard interface
>> *
>> * The device is a real keyboard that is used for normal input.
>> * Power-buttons and similar devices which report KEY_* events for
>> * non-keyboard buttons are not considered a keyboard in this sense.
>> * Only real keyboards and small extension devices (like external
>> * NumPads) belong here.
>> * The defined KEY_* events are not listed here as normally no
>> * mapping is applied. Libraries like libxkbcommon should be used
>> * for keyboard handling and mapping.
>> */
>> INMAP_CAP_KEYBOARD = (1 << 0),
>>
>> /**
>> * @INMAP_CAP_MOUSE
>> * Mouse interface
>> *
>> * The device is a real mouse with relative motion events. Motion
>> * is reported as REL_X/Y. Buttons are reported as BTN_0-BTN_9 or
>> * BTN_LEFT-BTN_TASK.
>> * Other mouse-like devices (especially those with absolute motion
>> * events) do not belong here.
>> */
>> INMAP_CAP_MOUSE = (1 << 1),
>>
>> /**
>> * @INMAP_CAP_TOUCHPAD
>> * Touchpad interface
>> *
>> * Touchpad devices belong here. But only those that are used as
>> * pointing devices and are intended to drive a mouse device. That is,
>> * the input is interpreted as relative motion, even though the
>> * reported data is normally absolute.
>> * Hence, touchscreens do not belong here. They are normally
>> * interpreted as absolute position input.
>> *
>> * For multitouch devices, libmtdev defines the protocol.
>
> kernel MT Procol B is a better term here, libmtdev merely converts A to B.

I actually meant that this includes all MT protocols (all A models and
B), that is, eveything that mtdev accepts. A driver which uses this
would then probably call mtdev to get B only. However, doing this
conversion here would be useless, I think, because we already have a
consistent interface with the 3 (or 4?) different MT protocols.

But I am also fine with calling mtdev directly here and defining this as MT-B.

>> *
>> * @TODO define the protocol (see the synaptics descriptions)
>> */
>> INMAP_CAP_TOUCHPAD = (1 << 2),
>>
>> /**
>> * @INMAP_CAP_TOUCHSCREEN
>> * Touchscreen interface
>> *
>> * Touchscreens are exactly the same as @INMAP_CAP_TOUCHPAD, but they
>> * are normally attached to a single monitor and interpreted as
>> * absolute positioning input.
>> * Therefore, they are not included in @INMAP_CAP_TOUCHPAD, but are
>> * a separate group instead. The reported protocol is the same, though.
>> *
>> * @sa INMAP_CAP_TOUCHPAD
>> */
>> INMAP_CAP_TOUCHSCREEN = (1 << 3),
>>
>> /**
>> * @INMAP_CAP_ACCELEROMETER
>> * Accelerometer interface
>> *
>> * Accelerometer devices report linear acceleration data as ABS_X/Y/Z
>> * and rotational acceleration as ABS_RX/Y/Z.
>> *
>> * @TODO this collides with ABS_X/Y of absolute pointing devices
>> *       introduce ABS_ACCELX/Y/Z
>> */
>> INMAP_CAP_ACCELEROMETER = (1 << 4),
>>
>> /**
>> * @INMAP_CAP_GAMEPAD
>> * Gamepad interface
>> *
>> * All standard gamepad devices belong here.
>> *
>> * @TODO define gamepad mapping (see linux-input at vger discussion)
>> */
>> INMAP_CAP_GAMEPAD = (1 << 5),
>>
>> /**
>> * @INMAP_CAP_JOYSTICK
>> * Joystick interface
>> *
>> * All standard joystick devices belong here.
>> *
>> * @TODO define joystick mapping
>> */
>> INMAP_CAP_JOYSTICK = (1 << 6),
>> };
>
> why are gamepads and joysticks different? buttons, a few axes that may or
> may not map to x/y and the rest is device-specific.
> this may be in the thread, but I still haven't gone through all msgs here.

Many ABS_* values are shared between joysticks and gamepads. So I
either map all gamepads to values that don't collide with joysticks or
I tell the application whether it's a joystick or gamepad. I think the
latter is easier.

>
> one missing section are graphics tablets but even there you could argue
> they're just buttons, x/y and a few extra axes that need client-specific
> handling.

I had DIGITIZER in here earlier, but removed it to keep it simpler. I
expect new types to be added over time, so I thought we could add
tablets later, too.

>> /**
>>  * @defgroup context Library Context
>>  * Managing library contexts
>>  *
>>  * A context object contains all top-level information. It can be shared
>>  * between multiple device objects and provides global infrastructure to them.
>>  *
>>  * @{
>>  */
>>
>> /**
>>  * @enum inmap_context_flags
>>  * Context creation and operation flags
>>  *
>>  * Different flags that control how a context is created and how it behaves.
>>  *
>>  * @memberof inmap_context
>>  * @sa inmap_context_new()
>>  */
>> enum inmap_context_flags {
>> /**
>> * @INMAP_CONTEXT_IGNORE_ENVIRONMENT
>> * Ignore environment variables
>> *
>> * The context object will not use any environment variables as
>> * default values if they are not explicitly set.
>> *
>> * @memberof inmap_context
>> */
>> INMAP_CONTEXT_IGNORE_ENVIRONMENT = (1 << 0),
>>
>> /**
>> * @INMAP_CONTEXT_MONITOR_CONF_FILES
>> * Monitor configuration files
>> *
>> * If set, the context will monitor configuration files for runtime
>> * modifications. This allows adapting to changes without restarting
>> * the application.
>> * Changes are not automatically applied. The application has to
>> * react to these events and retrigger device detection for all
>> * active devices.
>> *
>> * @memberof inmap_context
>> * @sa inmap_context_dispatch()
>> */
>> INMAP_CONTEXT_MONITOR_CONF_FILES = (1 << 1),
>
> tbh, I'd make that a required behaviour.

That would mean libraries that just want to enumerate/list devices
need this monitor. But I could at least make it the default and add a
"NO_MONITOR" flag.

Thanks!
David