Input and games.

[Pekka Paalanen]

Hi Todd,

you've provided lots of valuable information already. Unfortunately my
input is left as hand-waving, since I cannot dedicate to designing this
protocol myself (as in writing the XML spec).


On Mon, 29 Apr 2013 10:17:31 -0400
Todd Showalter <todd at electronjump.com> wrote:

> On Mon, Apr 29, 2013 at 4:15 AM, Pekka Paalanen <ppaalanen at gmail.com>
> wrote:
> 
> > a problem here is that to receive a wl_gamepad::connect event, you
> > first have to create a wl_gamepad protocol object, which is a bit
> > counterintuitive.
> >
> > A wl_gamepad protocol object should correspond to a single physical
> > device. So, we would have a wl_gamepad object per each contoller,
> > and you do not need the pad_index argument in any of the events.
> > This should be easier on the game programmer, too, since you can
> > attach user data to each protocol object in libwayland-client, and
> > use that in the callbacks without any global data.
> 
>     That's not necessarily how games are designed, though; I know that
> with our engine, input is represented internally as an array of
> gamepads.  It lets us do things like feed a pad with a stack of pad
> values to do things like demos and "replays".

Alright. The remaining reasons are still significant, IMHO: this is the
design pattern for Wayland input devices and even in general, and the
protocol needs to move less bytes when the pad_index is effectively
baked into the object id on the wire.

You can make your own pad_index and store it in the user data field, or
just store a pointer to your table. It's up to you, and you don't need
to deal with someone else's indices.

> > That leaves the question, where do we put the connect and disconnect
> > event equivalents. Below you mention also keyboards, so instead of
> > playing ad-hoc, let's use wl_seats the way they are designed.
> >
> > A wl_seat would need to grow a capability bit for wl_gamepad, and a
> > request to create a wl_gamepad object. When you use that request to
> > create a wl_gamepad, the first thing it does is send its
> > description: name and cookie in your proposal, as an extra event
> > type.
> >
> > That limits us to one wl_gamepad device per wl_seat, so a server
> > needs to create more wl_seats for more controllers. That shouldn't
> > be any problem, these seats would only have the gamepad capability
> > by default.
> 
>     This is I think where there's a potential problem.  Gamepads live
> in a somewhat more chaotic world than mice and keyboards; wireless
> ones have much shorter battery lives, and players are used to being
> able to unplug and plug them while playing.  It's not uncommon for
> someone to (say) play for a bit with a gamepad they don't like (maybe
> it was on sale), unplug it, and plug in one they like better.  Or drop
> the gamepad that ran out of batteries in the charger and pull out
> another.
> 
>     Players also expect to be able to add a gamepad part way through a
> game, at least for some games.
> 
>     So, gamepads can appear and disappear during the game's runtime,
> and the game needs to know that is happening.  There also need to be
> some heuristics about which gamepad is what player (or seat).

So you would rather handle all that in your game, than rely on the
display server to sort it out for you? The user would have to set up
each game, instead of just the display server. The display server could
also drive the player id indicators on some game controllers.

I can easily imagine a game controller configuration GUI for a display
server, where you can register game controllers and assign them to
different seats, just like you would do for keyboards and mice.

I don't really see why each game would need to reimplement this. Is it
just a habit, since there has not been any options?

Doesn't the game and the display server have the exact same problems?
Identifying devices, assigning them to players, ...

Except that when a display server does it, it can also be smart about
input dispatching to different simultaneous clients (games). Games
might be exclusive applications in general, but I don't think we should
encode that into the protocol.

> > If your gamepad actually had a keyboard, or maybe even a touchpad
> > (if that is supposed to be used as a pointer device), it would
> > simply be advertised as a standard wl_keyboard or wl_pointer on the
> > wl_seat. Each player would have their own wl_seat, and it is
> > obvious which keyboard belongs with which gamepad.
> 
>     That does solve that problem nicely.  It's somewhat of a corner
> case, though, so I wouldn't move mountains to solve it.
> 
> > Oh, and the disconnect event. The standard wl_seat way for that
> > seems to be a new capabilities event, with the gamepad bit unset.
> 
>     Ok.
> 
> > All this still leaves some details unsolved, like which Wayland
> > client should receive the gamepad events on a wl_seat? The one
> > having the keyboard focus? Probably. But what if a wl_seat has no
> > wl_keyboard or wl_pointer? Just the first client that creates a
> > wl_gamepad? What if you have two games running, and you want to
> > switch between them? Should a wl_gamepad have its own focused
> > surface attribute? How do you assign that focus? If you have other
> > input devices in addition to a gamepad on a wl_seat, how do you get
> > all their foci to the game, when the user wants it? How does the
> > user indicate he wants it?
> 
>     I think all gamepad input should be routed to whatever has focus
> or whatever has grabbed input.  I don't see a scenario where it makes
> sense to route different gamepads separately unless you're doing
> multiuser multihead (which I assume is the point of the wl_seat
> abstraction).

A wl_seat does not relate to any specific output. Each wl_seat on a
server simply shares all outputs with all other wl_seats.

If you want to do separate sessions, that is each user has his own
desktop, own input devices, and own outputs, then you pretty much run
one display server for each user.

wl_seats OTOH allow one to have several people collaborating on the
same session and desktop, or just one person who needs more than one
keyboard focus, for instance. This is one display server with several
wl_seats.

wl_seat is not really a seat in the physical sense. It may be better
thought via input devices and input foci. One wl_seat has zero or
one wl_keyboard, wl_pointer, etc. If you have several physical
keyboards, they all act as one. To have two keyboards independent, they
are assinged to different wl_seats. Then each keyboard can be typing
into a different window at the same time.

Focus is per wl_seat, and mostly per input device type. Keyboard and
pointer have their own foci, wl_touch does not have a focus at all in
the protocol.

So for games with multiple local players on the same screen, wl_seat
would be just a player id.

Does this clarify what I was talking about?


Thanks,
pq