Input and games.

Wed Apr 24 08:03:35 PDT 2013

On Wed, Apr 24, 2013 at 9:41 AM, Todd Showalter <todd at electronjump.com>wrote:

> On Wed, Apr 24, 2013 at 2:26 AM, David Herrmann <dh.herrmann at gmail.com>
> wrote:
>
> > That's a known problem that isn't easy to fix. Of course, we can
> > adjust the kernel driver, but this breaks applications that expect the
> > current behavior. The problem I see is that we never paid enough
> > attention how keys are mapped when adding kernel drivers and now we
> > have a mess of different mappings that cannot be fixed. You can try to
> > send patches to linux-input at vger.kernel.org, but chances are low that
> > they will get merged.
>
>     In the particular case of the ps3 controller, I have trouble
> believing anything is depending on the current behaviour, at least as
> I see it.  I haven't tried to use the data, but the button and axis
> mappings just look wrong.  The right stick is coming in as (ABS_Z,
> ABS_RZ), which is how the left and right shoulder triggers are mapped
> on the xbox controller.  The buttons have names like DEAD and "(?)",
> the latter of which looks to be evtest's name lookup failure string.
>
>     The more general problem, though, is that it means we have to
> treat every gamepad as unique, when they should be as ubiquitous and
> generic as mice.
>
>     Sitting next to me here is a Kensington Slimblade trackball.  It's
> my "mouse".  It has two axis control like a mouse, four buttons,
> pretends to have a scroll wheel (rotating the ball around the z axis
> clockwise or counterclockwise sends mouse wheel clicks), and generally
> works.
>
>     The top two buttons on it are mapped totally wrong on the wire
> when it talks over USB.  Rather than coming in as buttons 2 and 3, the
> stupid thing sets flags in an option field in the USB data packets.
> There's a quirk in the kernel driver to remap them to BTN_MIDDLE and
> BTN_SIDE, so as far as anything reading the evdev stream is concerned,
> it's just a mouse like any other, with four buttons and a wheel.
>
>     I can't do that with gamepads.  I should be able to.  The core
> idea of a gamepad (two sticks, two analog shoulder triggers, a dpad, a
> "home" button, a "start" button, two shoulder buttons, four face
> buttons opposite the dpad, and click on the two sticks) is universal
> enough at this point that it ought to be considered a basic system
> abstraction, like the core idea of a mouse.
>
> > Nevertheless, the problem is actually easy to fix in userspace. You
> > just need to create buttons mappings for the different devices. There
> > is no complex logic involved. It would be enough to have a bunch of
> > static tables indexed by input-device names which map the input
> > keycode to the correct/expected output keycode.
>
>     "easy to fix in userspace" rather ignores the deployment problem,
> and it also ignores the disconnect between the people adding hardware
> support and the people mapping it to something useful.
>
>     If they can't just be fixed in place, what I'd much rather see is
> one of two fixes:
>
> a) an ioctl() or some similar knob I can frob to say "deliver events
> in canonical form, please"
>
> b) have gampads produce *two* /dev/input/eventX entries, one for
> "Microsoft X-Box 360 pad" or whatever, and the other for "system
> gamepad"; deliver events in the current form over the named device,
> and in canonical form over the "system gamepad" device
>
>     Of the two solutions, I prefer the latter; it means I can walk
> down the list looking for "system gamepad" and only open those, but
> know that I'll get data I don't have to massage.  It also means that
> as soon as the kernel supports a new bit of hardware, it will Just
> Work. We won't have to deal with keeping an external library in sync.
> We won't have to try to examine the output of EVIOCGNAME() and try to
> figure out if this is one of the devices we support.  We won't fail to
> use an XBox 360 controller just because the device string said
> "MadCatz" instead of "Microsoft" and we'd never seen one of those in
> the wild and hadn't added it to our list of acceptable device name
> strings.  Or because we were using vendor IDs and didn't have that
> vendor ID on our list.
>
>     I think Linux actually has a decent shot in the next five years of
> becoming the OS for living room game and streaming media PCs (heck,
> we've got Valve sort-of onside, Microsoft's sitting unloved in the
> win8 pit, and Apple has been heading in the direction of more
> iOS/portable, not sit-down computing), but if it's going to happen
> then things like gamepads need to Just Work.  If we're at a place
> where the FAQ is telling people that in order to support their new
> Logitech MustThraster JoyBanger they need to pull version 0.19.7-r3 or
> later of libGamepadTrans.so from github, it all goes pear-shaped for
> most people fast.
>
> > But I cannot see a reason why a compositor should do this, though.
> > This can be easily put into a library and every game that needs it
> > performs the mappings. Table-mappings add a single memory-read which
> > shouldn't affect performance and clients can share the library.
> > The compositor isn't interested in joystick/gamepad events so my first
> > approach would be to let clients handle them.
>
>     I don't think the clients should be dealing with this, and neither
> should the compositor.  The data should be in useful format at its
> source.  If the source is producing hardware-specific data then it
> needs to be fixed as close to the source as is practically possible.
> Every layer the data travels through in the wrong format multiplies
> the amount of work that ultimately needs to be done to fix it.  Rather
> than having it remapped at its source, we're talking about every
> client that wants to work with the data having to load and use
> translation libraries to unfuck the events, and those translation
> libraries need to have a complete mirror of the kernel code so they
> know what they need to do in order to do the unfucking on a
> per-hardware-device basis.  That's clearly the most complex and
> error-prone way we could possibly tackle the problem.
>
>     If we can't get good data from the kernel, the we should massage
> it before it gets handed to clients.  Amongst other things, that
> prevents the input translation layer from being embedded in the
> clients and becoming yet another bit of legacy that can't be broken
> even though it's making everything more complex.  When we start
> getting sane input from the kernel, the translation layer can come out
> of Wayland/Weston, and the clients won't need to know.  Nothing will
> break.
>
> > Input device detection is a mess. The kernel provides EV_* flags for
> > each device, but doesn't tell us what kind of device that is. It makes
> > it really hard to distinguish device classes. udev tries to do that,
> > but it never worked as well as we hoped. A solution would be to add
> > input PROP_* flags for device classes to the kernel and read these
> > from user-space.
> >
> > So what I am trying to say is: you cannot rely on ABS_X to mean the
> > same for every device. It might be an accelerometer or a touchpad or a
> > touchscreen, ... A PROP_KEYBOARD, PROP_ACCELEROMETER, PROP_GAMEPAD,
> > ... property would help to distinguish the different device classes.
> > In each class, the different flags should mean the same.
>
>     This is what brought me here in the first place.  Linux is mostly
> actually a pretty good environment for game development, but there are
> a couple of really grim areas, and this is one of them (the other is
> sound, with opengl support as a quickly improving third).  Right now,
> to use gamepads on Linux, you either need to use libjsw and have a
> priori knowledge of what hardware is plugged in, or use evdev with
> root permissions and have a priori knowledge about how different
> gamepads map their controls.
>
>     A translation library just cans a snapshot of that a priori
> knowledge, adds one more system dependency the user and developer have
> to care about, and one more thing that might go wrong.
>
> > I'm currently looking into an interface that provides file-descriptors
> > for wl_keyboard/wl_mouse for clients. The FDs are muted (EVIOCMUTE
> > proposed on linux-input by krh) while clients are inactive and unmuted
> > when they get input focus. This is basically a performance boost
> > because input events no longer pass through the compositor.
> > However, this mechanism could be easily used to forward any other
> > input fd to clients. A wl_gamepad interface could be just empty except
> > for this FD-passing logic.
>
>     Being able to talk to the gamepads through evdev without elevated
> permissions would definitely be a step forward.
>

Hi all,
I hope you'll allow me to chip in just a bit here.  As a disclaimer, I
haven't read the entire thread but I think I have a decent idea of what's
gone on.

Personally, I would like to see some sort of a wl_joypad interface that
isn't just an FD passthrough.  Sure, it will probably be optional and
clients shouldn't assume it, but I think it would still be good to have it
there.  Part of the reason is that, as of now, the wayland protocol isn't
terribly restricted to standard Linux and I would like to avoid things that
require Linux underneath such a an FD that's expected to be a Linux evdev.
There are a couple of reasons for this.

One reason is if we want to do network forwarding.  Now, instead of simply
providing a forwarded version of the wl_joypad we have to forward the evdev
and possibly do some translation of the stream in order to make everthing
happy (I'm not sure what all this would involve).  If the client expects to
be able to do any ioctls on said stream things would get particularly
interesting.

Second is that events may not come directly from the kernel.  Once my
Android compositor app gets beyond just displaying the simple-shm client,
I'm going to want to forward input events including mouse, touch, virtual
keyboard, USB keyboard, and gamepad events.  The problem is that NONE of
those events come from evdev.  They all come from the Android event system
and need to be translated into something useful.

I realize that my little Android project shouldn't be the sole driver of
protocol decisions, but I don't think that is the only case where game
controller events would come from something that's not evdev.  As another
example, people have talked about Wayland on FreeBSD; how does FreeBSD
handle game controllers?  Can we assume that some sort of evdev fd passing
will work there and on Linux in any sort of reasonable way?

Anyhow, there's my 2 cents worth.
--Jason Ekstrand

>
>                                        Todd.
>
> --
>  Todd Showalter, President,
>  Electron Jump Games, Inc.
> _______________________________________________
> wayland-devel mailing list
> wayland-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/wayland-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/wayland-devel/attachments/20130424/c2517028/attachment-0001.html>