Input and games.

[Todd Showalter]

On Sun, Apr 21, 2013 at 8:14 AM, Martin Minarik
<minarik11 at student.fiit.stuba.sk> wrote:

> I disagree with inventing a new protocol. The joypad
> devices are generally already supported in the linux
> kernel[1].

    They are, but from a developer point of view it's not pretty; it's
been a while since the last time I messed with evdev (basically after
getting frustrated with the limitations of libjsw), but IIRC:

- you need elevated permission to open the device -- risky in
something as complex as a game, especially when it may well be running
mods, scripts or data from untrusted sources (ie: found on a forum)

- the data comes in as EV_KEY, EV_REL and EV_ABS messages, which is
fine, but I found no programmatic way to discover the structure of the
device -- I don't know how relative axis 3 being set to 0.5 maps onto
what the player is doing with the actual device

    So, not ideal.  It's a decent basis for a higher level system,
assuming there's some way to query device type and determine how the
various parts of the device actually map onto the physical controller,
but something has to supply that higher level system, and right now
it's the game.

    The caveat here is that I last touched libjsw and evdev a few
years back; perhaps there have been improvements or someone wrote the
higher level support code and I missed it.

    On the permissions side of things, though, the ideal is that a
game be able to setuid to something with nearly no permissions at all
and still be able to receive gamepad, keyboard and mouse messages.
Not all games will want to do that; some things will want more
filesystem access than just a save directory, some games will want to
be able to bind network ports and so forth.  But ideally, for a game
that is local-only and only needs to access a small directory to save
its state, it should be able to have it's own unprivileged userid that
it runs as without sacrificing gamepad input.

> The protocol is similiar to evdev. We may be able to send
> it over the wayland protocol to the applications in a
> straightforward way.

    This is workable, but identifying how the parts map to the device
is still a problem.  With a keyboard, you get keysyms that tell you
what keys the messages correspond to.  With a mouse, you get an
explicitly labelled 2d vector for the pointer and numbered buttons in
a mask that you can directly map onto the hardware without worry.
With gamepads, unless things have changed, the axis and button values
come in unchanged from the order they were on the hardware, which
means that you have no idea which control a given button or axis maps
to.

    That's the reason I'm suggesting a standardized layout, since
devices now make that feasible.  That's not to say that the standard
layout needs to be transmitted with every message.  I'd be perfectly
content if it was still component-based messages, as long as enough
information was available.  To put it in (cut down) X11 terms:

typedef struct {
   [... protocol metadata ...]

   int type; // press or release
   unsigned int keycode; // detail
} GamepadKeyEvent;

typedef struct {
   [... protocol metadata ...]

  int type; // stick
  int index; // 0 is left stick, 1 is right stick
  float x, y; // deflection [-1.0f .. 1.0f]
} GamepadStickEvent;

typedef struct {
  [... protocol metadata ...]

  int type; // axis
  int index; // 0 is left trigger, 1 is right trigger
  float x; // deflection [0.0f .. 1.0f]
} GamepadAxisEvent;

    So, it could support arbitrary controllers with more than 2
sticks, but the user would be guaranteed that the first two sticks map
to the standard left and right sticks on the player's game controller.
 Likewise, it could support more than two single-axis values, but the
common case is standardized.  Most games won't need any extra glue
code to work with a scheme like this.

    The fun bit is the keysyms for the buttons.  Everyone labels their
buttons differently.  While most controllers have a "start" button
somewhere, the face buttons (ie: the ones games usually want to tell
you to press...) are a mess.  Clockwise from the top, we have:

PS*: green triangle, red circle, blue x, pink square
XBox: yellow Y, red B, green A, blue X
Wii Classic: solid x, clear a, clear b, solid y
Ouya: yellow Y, red A, green O, blue U

    So, I think a sane protocol can assume four face buttons, since
that seems to be standard since Sega stopped making hardware, but
they'll need  hardware independent names; top/bottom/left/right or
compass directions or the like.  The "start" button maps across
hardware.  Some systems have "home" or "select" buttons that can be
mapped, and most systems have shoulder triggers that can activate as
buttons.  Many of the systems also have "click" on their analog
sticks, and obviously dpads should map cleanly across hardware.

    So, yes, an event-based system that delivers gamepad events
piecewise is perfectly workable, but for it to be useful to games it
needs to provide some way for the game to map that information to
standard controls.  This leaves aside things like the wiimote and the
razer device that were mentioned earlier, but those open up all sorts
of cans of worms.  The razer device sounds like it has a completely
nonstandard data format (sending the info in quats is excellent; don't
get me wrong, but AFAIK nothing else does that), *all* motion devices
I've messed with have serious noise and lag problems, and motion-based
devices make heavy use of gesture recognition.

    Plus, with the wiimote, you get into a weird space where every
player has their own pointer on the screen, and all sorts of basic
assumptions get broken like cheap glass.  Things like "how focus
works".  You sort of have the same problems with touch, but touch has
less context and doesn't tend to be multiple people competing.  The
wiimote is exactly like giving multiple people their own mouse
pointers on the same screen.

    It's only really the standard gamepad case that I would really
like to see solved.  The rest is inessential, though nice to have.

                                           Todd.

--
 Todd Showalter, President,
 Electron Jump Games, Inc.