[RFC libinput 1/2] Add a "buttonset" interface for button-only devices

[Jason Gerecke]

On Wed, Feb 18, 2015 at 5:29 PM, Peter Hutterer
<peter.hutterer at who-t.net> wrote:
> On Wed, Feb 18, 2015 at 01:57:17PM -0800, Jason Gerecke wrote:
>>
>> On 2/16/2015 9:11 PM, Peter Hutterer wrote:
>> >On Mon, Feb 16, 2015 at 08:13:01AM +0100, Hans de Goede wrote:
>> >>Hi,
>> >>
>> >>On 16-02-15 04:50, Peter Hutterer wrote:
>> >>
>> >><snip>
>> >>
>> >>>ok, I've played around with the ideas in this thread and discussed it with
>> >>>Benjamin this morning. Short summary: I think we should go with the original
>> >>>patch, with an optional extension for numbered axes later.
>> >>>
>> >>>I tried the enum spacing first, providing an API where  we have
>> >>>    LIBINPUT_BUTTONSET_AXIS_TYPE_COUNT = 512
>> >>>    LIBINPUT_BUTTONSET_AXIS_RING = 512
>> >>>    LIBINPUT_BUTTONSET_AXIS_STRIP = 1024
>> >>>
>> >>>so the caller can use code like
>> >>>    libinput_event_buttonset_get_axis_value(event,
>> >>>                                    LIBINPUT_BUTTONSET_AXIS_RING + 2);
>> >>>to get the third axis. switching event-debug showed the issues with this
>> >>>approach: to effectively go through the axes, the client needs two loops,
>> >>>one for the type one for the number and mask the enum into a type and a
>> >>>number component, something that's prone to bugs.
>> >>>
>> >>>Next attempt was to split value and axis number explicitly in the API:
>> >>>    libinput_event_buttonset_get_axis_value(event,
>> >>>                                            LIBINPUT_BUTTONSET_AXIS_RING,
>> >>>                                            2);
>> >>>Better than the above as it's less error-prone.
>> >>>It still doesn't remove the above issue though, nested loops everywhere to
>> >>>access the type and the number.
>> >>>event-debug is a special case in that it doesn't care about the content and
>> >>>cares more about listing/printing everything. However I suspect that any
>> >>>generic toolkit will require the same.
>> >>>
>> >>>[Internally both changes are a bit of a nightmare as both would require some
>> >>>rewriting, but that's solveable]
>> >>>
>> >>>So the question is now: what does this gain us? better handling of truly
>> >>>generic devices with random axes. Which leads into the next question: what
>> >>>are random axes?
>> >>>The above approach is over-engineered because there's a group of axes
>> >>>that only exists once. I don't think there's a device with two X axes on the
>> >>>same device for example. Likewise, there are axes that have more use being
>> >>>semantically labelled than just numbered. The ring axes are a good example,
>> >>>the current RING and RING2 naming is bad, we should name it RING_LEFT,
>> >>>RING_RIGHT instead.
>> >>>
>> >>>Other than say a mix table we'll likely find semantic naming for a majority
>> >>>of the device. For the other devices, we can use a hybrid approach:
>> >>>leave the current API to use enums but in the future, when we require more
>> >>>flexible axes or devices with multiple identical axes we can add calls like:
>> >>>    libinput_device_buttonset_has_axis_by_offset(event, axis_number);
>> >>>    libinput_device_buttonset_get_axis_type_by_offset(event, axis_number);
>> >>>    libinput_event_buttonset_get_axis_value_by_offset(event,
>> >>>                                                      axis_number);
>> >>>
>> >>>etc. this would require a duplication of the buttonset API with a by_offset
>> >>>prefix but is otherwise straightforward and leaves the most expressive API
>> >>>as the default one.
>> >>>
>> >>>We also discussed making an axis an object (struct libinput_buttonset_axis)
>> >>>and providing a get_axis(enum) and a get_axis_by_offset(number) call. That
>> >>>object could've then be used in the API. I don't think it's a good idea to
>> >>>do this, it adds complexity (ref, unref, userdata) and effectively reduces the
>> >>>API to magic numbers/variable names again.
>> >>>
>> >>>so long story short, I think we should go with the enum after all and leave
>> >>>numbered/offset axes for the future when we need it.
>> >>>
>> >>>>And I do still believe we need an UNKNOWN axis, at XDC we were talking about these
>> >>>>button boxes people could buy to use with midi-apps, which were just a bunch
>> >>>>of dials, I don't want to do a database of those, so for such a device we
>> >>>>should just end up saying this is a buttonbox with UNKNOWN axis, which when
>> >>>>absolute must be used with get_transformed only, and the deltas are in an
>> >>>>unknown unit, but we can cross that bridge when we get there.
>> >>>once the axis is labelled once, it effectively becomes ABI. if we label
>> >>>something as unknown and then fix it in a later version of libinput we will
>> >>>break things. so we should hide anything we don't know enough about.
>> >>>Nothing stops us from adding a LINEAR_ACTUATOR, MIDI_DIAL, etc. though.
>> >>Hmm, I still have the feeling the end result is going to be way less generic
>> >>then what we had in mind when we introduced the buttonset concept, it seems
>> >>that you're more solving the tablet buttons problem here then introducing
>> >>a general buttonset API. But if you and Benjamin believe that this is the
>> >>right way forward lets go with this, we can always extend the API later.
>> >yeah, tbh the tablet case is the most obvious for now, it's likely to be the
>> >prime use-case for this interface. A truly generic interface with just
>> >numbered axes is IMO a lot easier to tack on top than the other way round.
>> >
>> >Cheers,
>> >    Peter
>> I'm also somewhat wary of proposed semantic interface. Sure, if the controls
>> had silkscreened labels indicating what they do, then it'd make a lot of
>> sense. Generally speaking though, the rings/strips/buttons only have
>> semantic meaning within a particular context (e.g. a running application or
>> user preference). The "_LEFT" and "_RIGHT" suffixes don't provide
>> particularly more insight about function than a numeral, it just gunks up
>> the naming. The controls really are more akin to MIDI button boxes than you
>> seem to credit, and the 27QHD ExpressKey remote moves even further to that
>> territory. I wonder if Wacom will ever re-visit [1] :D
>>
>> [1]: http://www.thecoolist.com/wacom-nextbeat-dj-controller/
>
> my main goal is to make sense of the various bits that come out of the
> device so that the next layer in the stack doesn't need a huge amount of
> knowledge about a specific tablet to process it any further. or at the very
> least take the guesswork out of the interface.
>
> so there are two specific types of devices that we look at: once where
> where we can make sense of the data and one where we can't. for tablet
> devices we can make sense of it and provide that to the caller.
>
> The nextbeat is actually a good example: the HW shows what each controller
> is, so we could export them as what it is, e.g. AXIS_SLIDER_CROSSFADE,
> AXIS_SLIDER_VOLUME etc. if a caller then wants to use them as generic
> sliders or something that's fine - dropping information is always easier
> than adding it.
>
Absolutely.

> take the example of the 24HD: we can provide two rings but without knowing
> about the tablet specifically you don't know what the two rings mean.
> LEFT/RIGHT helps a bit here (or EAST/WEST if we want to adopt the newer
> joystick button naming from the kernel).
> this also means that we can handle left-handed mode transparently inside
> libinput, the LEFT wheel is always the left wheel, even if the tablet is
> upside down (sorting out the buttons gets a bit tricky, I admit).
>
> libinput being where it is in the stack that doesn't mean that it translates
> to the actual application anyway. in the end, the actual mapping happens
> somewhere inside control-center/mutter/gnome-settings-daemon.
>
I suppose I just worry about a potential explosion of
almost-but-not-quite-identical names. Not because enumerating them is
difficult, but because each has to be treated as a special case even
if the only qualitative difference is something created by the
industrial designer. Perhaps one day they decide it'd be nifty to have
three color-coded touchstrips; adding definitions for STRIP_LEFT_RED,
STRIP_LEFT_YELLOW, and STRIP_LEFT_BLUE is easy enough but that's three
more cases to not forget to sprinkle throughout the code.

[re-attached Carlos' latest reply]
On Thu, Feb 19, 2015 at 3:35 AM, Carlos Garnacho <carlosg at gnome.org> wrote:
> On Thu, Feb 19, 2015 at 2:29 AM, Peter Hutterer
> <peter.hutterer at who-t.net> wrote:
>> On Wed, Feb 18, 2015 at 11:52:18PM +0100, Carlos Garnacho wrote:
>>> I'm not sure left/right specifically is something we want to tell
>>> about at this level. With tablet support in mind, I would expect a
>>> left-handed mode for buttonsets too, so rings/strips would flip side,
>>> be inverted, etc. We can sure flip the axes internally in libinput,
>>> but sounds inconsistent if buttons aren't to be actually flipped.
>>
>> I do intend to flip the buttons, once I figure out how :)
>>
>>> As for this semantical approach, are we sure the input_event codes
>>> won't overlap across different devices that might qualify here? It
>>> could be potentially confusing otherwise. TBH I share a bit the
>>> concern with Jason here, most probably all apps dealing with this will
>>> want their own dialogs and methods to map actions anyway.
>>
>> The lack of precision in the input_event codes is one reason we have different
>> enums for axes in the tablet interface already. we internally discussed
>> replacing (or augmenting) the current linux/input.h with libinput specific
>> ones but chose that this is not a bridge to cross just yet.
>>
>> I'm not sure I know what you mean with the overlap problem though, can you
>> expand?
>
> I mean, for example the intuos5 ring is advertised as ABS_WHEEL
> (meaning RING_LEFT on right-handed mode), picking on the nextbeat
> example again (and knowing nothing about it TBH, the example is made
> up and extreme), what if such device is partially recognized by
> libinput as a buttonset and advertises that axis but it actually means
> "treble equalization on track B"? (which by looking to the device
> images, isn't neither a ring, nor at the left)
>
> You can catch up reactively to this, and map internally in libinput
> the semantical meanings for each axis based on the device, but there's
> room for user confusion in the mean time. IMHO, a get_axis_type call
> that returns strip/ring/knob/whatnot and admits it's a best effort
> guess would allow for some generic treatment above libinput if we
> change our guesses over time.
>
> Even if we leak semantic labels all through the protocol, I'm not sure
> at which level would these make for better self-explaining code. I
> remember from LGM last year, when demoing the GNOME tablet button
> mapping, I got quite some "I want to borrow/steal this for $PAINT_APP"
> comments. It sounds likely to me that, given the case, they'd just map
> the axis number to whatever action, regardless of the enum value name.

Maybe a system of attributes could be used to get around having to
deal with updating multiple cases any time a new axis type is added? A
get_axis_type call to figure out how the axis is physically
implemented, a get_axis_location call to figure out if it should be
swapped for left-handed operation, etc.

Jason
---
Now instead of four in the eights place /
you’ve got three, ‘Cause you added one  /
(That is to say, eight) to the two,     /
But you can’t take seven from three,    /
So you look at the sixty-fours....