Input and games.

[Pekka Paalanen]

On Wed, 24 Apr 2013 08:26:19 +0200
David Herrmann <dh.herrmann at gmail.com> wrote:

> Hi Todd
> 
> On Tue, Apr 23, 2013 at 4:12 PM, Todd Showalter <todd at electronjump.com> wrote:
> > On Tue, Apr 23, 2013 at 7:25 AM, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> >
> >> what you describe here is very much a keymap-like database for game
> >> controllers: translating from button and axis indices to labels or
> >> symbols. However, having a brief chat with Daniel Stone, it seems we
> >> should not need these.
> >>
> >> Take a look at /usr/include/linux/input.h
> >>
> >> There you find definitions for BTN_A, BTN_X, BTN_START, ABS_X, ABS_Y,
> >> ABS_RX, ABS_RY, ABS_HATnn, and many more. The kernel evdev interface
> >> should alreay be giving out events with the correct label, so we would
> >> not need any mapping.
> >>
> >> Are you saying that the kernel gives out the labels wrong? If so, this
> >> should be fixed in the kernel drivers. One thing less to care about in
> >> Wayland. We "just" need to write the protocol for these devices, the
> >> labels should be already there.
> >
> >     I'm not saying the labels are wrong; I assume they are correct.
> > The problem is that the labels are hardware-specific, at least for the
> > buttons.  That said, it looks like the axis values are being properly
> > labelled, which means we're way closer to sane behaviour than we were
> > last time I looked into this.
> >
> >     I grabbed evtest and ran it with three devices; an xbox
> > controller, an xbox 360 controller, and a ps3 controller.  The
> > results:
> >
> > xbox:
> >     - button
> >        A B X Y START THUMBL THUMBR
> >        Z (white button)
> >        C (black button)
> >        SELECT (back button)
> >     - axis
> >        ABS_X ABS_Y ABS_Z
> >        ABS_RX ABS_RY ABS_RZ
> >        ABS_HAT0X ABS_HAT0Y (dpad)
> >
> > xbox 360:
> >     - button
> >        A B X Y START THUMBL THUMBR
> >        TL  TR
> >        MODE (home button)
> >        SELECT (back button)
> >     - axis
> >        ABS_X ABS_Y ABS_Z
> >        ABS_RX ABS_RY ABS_RZ
> >        ABS_HAT0X ABS_HAT0Y (dpad)
> >
> > ps3:
> >   - button
> >     TRIGGER  THUMB  THUMB2
> >     TOP TOP2 PINKIE BASE  BASE2
> >     BASE3  BASE3  BASE4  BASE5
> >     BASE6  DEAD  TRIGGER_HAPPY17
> >     TRIGGER_HAPPY18  TRIGGER_HAPPY19
> >   - axis
> >     ABS_X  ABS_Y  ABS_Z  ABS_RZ  ABS_MISC
> >
> >     The xbox controller and the xbox 360 controller are more or less
> > the same; the 360 controller has a couple of shoulder buttons instead
> > of a the black and white buttons, and (somewhat oddly) the "back"
> > buttons come in as "select", but that's workable.
> >
> >     It all rather goes pear-shaped when we get beyond that, though.
> > The PS3 controller, while physically quite similar to the other two,
> > even down to the placement of controls and how the controls are
> > clustered, comes in completely differently.  There is not a single
> > button in common between the PS3 controller and the XBox controllers
> > as reported by evdev, despite the PS3 controller having buttons
> > physically labelled "start" and "select", plus direct equivalents to
> > many of the XBox 360 controller's parts (ie: TL, TR, MODE, ABS_HAT0X,
> > ABS_HAT0Y, ABS_RX, ABS_RY...).
> 
> That's a known problem that isn't easy to fix. Of course, we can
> adjust the kernel driver, but this breaks applications that expect the
> current behavior. The problem I see is that we never paid enough
> attention how keys are mapped when adding kernel drivers and now we
> have a mess of different mappings that cannot be fixed. You can try to
> send patches to linux-input at vger.kernel.org, but chances are low that
> they will get merged.
> 
> Nevertheless, the problem is actually easy to fix in userspace. You
> just need to create buttons mappings for the different devices. There
> is no complex logic involved. It would be enough to have a bunch of
> static tables indexed by input-device names which map the input
> keycode to the correct/expected output keycode.
> 
> But I cannot see a reason why a compositor should do this, though.
> This can be easily put into a library and every game that needs it
> performs the mappings. Table-mappings add a single memory-read which
> shouldn't affect performance and clients can share the library.
> The compositor isn't interested in joystick/gamepad events so my first
> approach would be to let clients handle them.
> 
> >     The PS3 controller also has several "(?)" entries for buttons and
> > axis values, and also appears to have (if I understand correctly) a
> > bunch of codes for a multitouch panel?  I couldn't tell you what the
> > right or left stick axis values are in the above, because though I did
> > build my kernel with ps3 controller support, and evtest did see it and
> > dump the supported event list, I get no events logged from... ah, ok,
> > I have to hit the PS button to get it to actually work.  And now
> > there's a torrent of what I assume is accelerometer data coming in on
> > "(?)" events.
> >
> >     It turns out the left stick is ABS_X and ABS_Y, and right stick is
> > ABS_Z and ABS_RZ.  I suspect this is just broken somehow.  Maybe the
> > ps3 gamepad kernel driver is still a work in progress?  But this is
> > the kind of thing I was talking about; the data I get from a ps3
> > gamepad is mapped totally differently from the data I get from an xbox
> > gamepad, so from a game point of view, even if all I want is a
> > joystick, a jump button and a shoot button, I still have to care what
> > particular kind of gamepad the player has plugged in because I'm going
> > to get completely different button messages depending on what kind of
> > pad is plugged in.
> 
> Input device detection is a mess. The kernel provides EV_* flags for
> each device, but doesn't tell us what kind of device that is. It makes
> it really hard to distinguish device classes. udev tries to do that,
> but it never worked as well as we hoped. A solution would be to add
> input PROP_* flags for device classes to the kernel and read these
> from user-space.
> 
> So what I am trying to say is: you cannot rely on ABS_X to mean the
> same for every device. It might be an accelerometer or a touchpad or a
> touchscreen, ... A PROP_KEYBOARD, PROP_ACCELEROMETER, PROP_GAMEPAD,
> ... property would help to distinguish the different device classes.
> In each class, the different flags should mean the same.

*sigh*

So, we're back to the mapping library approach.

But if there will be a mapping library, maybe it could as well do the
device class detection? In that case, it would not be a huge problem to
add, say, evdev device name to device class table there, and if a
device is not there, fall back to heuristics on properties and event
types... eh, except it would need to detect not only game controllers
but also *all* other kinds of devices to avoid assuming they are game
controllers too. Bleh.

And class detection really belongs in the server. We've already had our
share of detection pain with just touchscreens and trackpads.

The mapping is client business, especially if we just give an evdev fd,
and it seems to align well with existing Wayland conventions like
keyboard handling.

> >> The current behaviour can be checked with evtest:
> >> http://cgit.freedesktop.org/evtest/
> >> Was that what you used to check the controller events?
> >
> >    Previously, I'd been dumping data from the libjsw interface, and
> > then dumping data by going directly through evdev.  This time I used
> > evtest.
> 
> I'm currently looking into an interface that provides file-descriptors
> for wl_keyboard/wl_mouse for clients. The FDs are muted (EVIOCMUTE
> proposed on linux-input by krh) while clients are inactive and unmuted
> when they get input focus. This is basically a performance boost
> because input events no longer pass through the compositor.
> However, this mechanism could be easily used to forward any other
> input fd to clients. A wl_gamepad interface could be just empty except
> for this FD-passing logic.

Yeah, that crossed my mind, too. Excellent to hear that you are working
on it.

To me it looks like the server must do device class detection anyway,
and then it can advertise input devices based on class.

I do wonder how game controllers would fit into the wl_seat scheme. The
idea of a wl_seat is combine all physical input devices of the same
class, tagged for the same wl_seat, into just one class instance
per class for clients. But that won't work with evdev fd passing
anyway, let alone with different kinds of game controllers it seems.


Thanks,
pq