Input and games.

[Pekka Paalanen]

On Fri, 3 May 2013 17:42:20 +0100
Daniel Stone <daniel at fooishbar.org> wrote:

> Hi,
> 
> On 3 May 2013 08:17, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> > On Thu, 2 May 2013 19:28:41 +0100
> > Daniel Stone <daniel at fooishbar.org> wrote:
> >> There's one crucial difference though, and one that's going to come up
> >> when we address graphics tablets / digitisers too.  wl_pointer works
> >> as a single interface because no matter how many mice are present, you
> >> can aggregate them together and come up with a sensible result: they
> >> all move the sprite to one location.  wl_touch fudges around this by
> >> essentially asserting that not only will you generally only have one
> >> direct touchscreen, but it provides for multiple touches, so you can
> >> pretend one touch each on multiple screens, are multiple touches on a
> >> single screen.
> >
> > Right. Could we just say that each such non-aggregatable device must be
> > put into a wl_seat that does not already have such a device?
> > Or make that an implementors guideline rather than a hard requirement
> > in the protocol spec.
> 
> *shrug*, it really depends.  If we're going to say that gamepads are
> associated with focus, then they have to go in a _specific_ wl_seat:
> the focus is per-seat, so if we're saying that clicking on a window
> with your mouse to focus it (or Alt-Tabbing to it) also redirects
> gamepad events there, then the gamepad needs to be part of _that_ seat
> which changed the focus.  Remember that we can have multiple focii per
> the protocol (though the UI for that gets very interesting very
> quickly).
> 
> If they're unaffected by the focus - which they would be if they're
> just going into random new wl_seats - then they shouldn't be in
> wl_seat just because it's the container we have for input devices
> right now, they should have their own interfaces.  Which really means
> wl_gamepad_manager which, when bound to, advertises new_id
> wl_gamepads.
> 
> tl;dr: wl_seat has a very specific meaning of a set of devices with
> one focus, please don't abuse it.

I'm not too clear on what it is.

In a wl_seat, we have one kbd focus, and one pointer focus. These
two are unrelated, except sometimes some pointer action may change
the kbd focus. Most of the time, they have no relation.

I was thinking of adding a third one: the gamepad focus. It could
be independent from kbd and pointer foci, or maybe it is assigned
with the kbd focus. Or maybe the gamepad focus is assigned to the
surface having any wl_seat's the kbd focus, whose client has bound
to the gamepad.

In any case, we have the fundamental problem: which client gets the
gamepad events at a point in time?

There can be several clients bound to any gamepad, and the target
(focus) must be switchable intuitively.

Is it wrong to think a wl_seat as a user--a player, that may have a
gamepad?

It's just too tempting for me to think that each player corresponds
to a particular wl_seat.

> >> The gamepad interaction doesn't have this luxury, and neither do
> >> tablets.  I don't think splitting them out to separate seats is the
> >> right idea though: what if (incoming stupid hypothetical alert) you
> >> had four people on a single system, each with their own keyboards and
> >> gamepads.  Kind of like consoles are today, really.  Ideally, you'd
> >> want an association between the keyboards and gamepads, which would be
> >> impossible if every gamepad had one separate wl_seat whose sole job
> >> was to nest it.
> >
> > So... what's wrong in putting each keyboard into the wl_seat where it
> > belongs, along with the gamepad?
> 
> In that case, yes, we would have wl_seats with one wl_keyboard and
> multiple wl_gamepads.
> 
> >> I think it'd be better to, instead of wl_seat::get_gamepad returning a
> >> single new_id wl_gamepad, as wl_pointer/etc do it today, have
> >> wl_seat::get_gamepads, which would send one wl_seat::gamepad event
> >> with a new_id wl_gamepad, for every gamepad which was there or
> >> subsequently added.  That way we keep the seat association, but can
> >> still deal with every gamepad individually.
> >
> > It would be left for the client to decide which gamepad it wants from
> > which wl_seat, right?
> >
> > Do we want to force all clients to choose every non-aggregatable device
> > this way?
> 
> No idea. :)
> 
> > Essentially, that would mean that wl_seat are just for the traditional
> > keyboard & mouse (and touchscreen so far) association, and then
> > everything else would be left for each client to assign to different
> > wl_seats on their own. This seems strange. Why do we need a wl_seat
> > then, why not do the same with keyboards and mice?
> >
> > Oh right, focus. You want to be able to control keyboard focus with a
> > pointer. Why is a gamepad focus different? Would all gamepads follow
> > the keyboard focus? If there are several wl_seats with kbd & ptr, which
> > keyboard focus do they follow? What if the same gamepad is left active
> > in more than one wl_seat? What if there is no keyboard or pointer, e.g.
> > you had only a touchscreen and two gamepads (say, IVI)?
> >
> > And then replace every "gamepad" with "digitizer", and all other
> > non-aggregatable input devices, and also all raw input devices via
> > evdev fd passing. The fd passing I believe has similar problems: who
> > gets the events, which wl_seat do they follow.
> >
> > This is a new situation, and so many open questions... I just continued
> > on the exitisting pattern.
> 
> Yeah, it really all depends on these questions.  But intuitively, I'd
> say that gamepads should follow a seat's focus, which means expanding
> wl_seat to be able to advertise multiple gamepads.  Even on touch, we
> still have wl_touch as part of wl_seat, driving the focus.  And I
> don't think a gamepad could ever be a part of multiple seats; perhaps
> it could be shifted between seats if necessary, but this is a problem
> we already have with keyboard, pointer and touch today.  And you don't
> need to deal with that in the protocol: just have the compositor
> destroy the device and create a new one in the new seat.

Right, so if every wl_seat advertises every gamepad, how should the
server route the gamepad input events? Should it be allowed to
duplicate the events for every active wl_seat-binding of a gamepad?

Or, do we want to have a unique input event destination, and never
duplicate events to several destinations?

If every wl_seat advertises every gamepad, how do we avoid having a
gamepad part of several wl_seats at the same time? When any client
binds to any gamepad on any wl_seat, will the server then make
that gamepad unavailable in all other wl_seats?

We do not have this problem with keyboards and pointers, because
keyboards and pointers are pre-assigned to wl_seats, and they do
not change by client actions, nor do clients assume they can access
all keyboards and pointers via the same seat. Unless, we think
about having two separate pointers, and just one physical keyboard,
and wanting to type for both pointers, one at a time. I don't
recall any discussion of that, and I've just assumed it won't be
implemented.

**

A very simple solution could be this (leveraging from your ideas):

wl_gamepad_manager:

A global interface, where you register wl_surfaces, and which
advertises all gamepads in the session.

wl_gamepad:

Interface representing a single gamepad. Input events from the
physical device are routed here, when any of the registered
wl_surfaces have keyboard focus.

There are some problems:
- how to associate keyboards etc. with a player (gamepad)?
- which seat's keyboard focus to follow?
Seems that we have to involve the wl_seat somehow.

Could this be a way forward?

Or maybe wl_gamepad_manager is not a global itself, but when
created, it gets bound to a specific wl_seat? Solves one problem,
but introduces other ones...


Thanks,
pq