Gamepad focus model (Re: Input and games.)

[Jason Ekstrand]

On Tue, May 14, 2013 at 10:18 AM, Daniel Stone <daniel at fooishbar.org> wrote:

> Hi,
>
> On 14 May 2013 05:49, Rick Yorgason <rick at firefang.com> wrote:
> > Okay, so the most common configuration would be one seat with a mouse,
> > keyboard, and gamepad. If you only ever play single player games, this is
> > all you would see.
> >
> > When you have multiplayer games, you just want to be able to launch the
> game
> > and have everybody automatically in it. You don't want your extra
> players to
> > have to think about focusing the app or anything like that.
>
> Yes, 100%.
>
> > That's the crux of the problem: how do you have one focus for multiple
> > players? The many-gamepads-per-seat model solves this, but it has a
> couple
> > problems:
> >
> > 1) Some compositors may put every gamepad in one seat, and some
> compositors
> > may assign one gamepad per seat. The game programmer has to support both
> > code paths. And unfortunately, when game programmers inevitably neglect
> to
> > do that, the gamers are going to blame the compositors rather than the
> > games. Wayland will get a reputation as a display server where you have
> to
> > configure your gamepads differently for different games.
>
> Compositors can implement maximise badly too, or multi-display
> support.  But it's not an argument to push the complexity out.  It's
> an argument for not running broken compositors.
>
> > 2) Gamepads aren't the only scenario where you want to have one focus for
> > multiple players.
> >
> > Here's a couple scenarios that are complicated by breaking the seat ==
> user
> > model:
>
> It's not seat == user, it's seat == focus.
>
> > A) You're playing split-screen Halo. Two users have a mouse/keyboard, and
> > two users have gamepads.
> >
> > In the seat == user world, we know how this is set up: seat 1 has a
> > mouse/keyboard, seat 2 has a mouse/keyboard, seat 3 has a gamepad, and
> seat
> > 4 has a gamepad.
>
> We know how it's set up, but we don't know how it works.  How does the
> compositor know to focus 3 and 4 at Halo, rather than at the browser
> or whatever else is running at the time? You can't have the keyboard
> and mouse tell you, because they're different seats.  So it's just
> manual then? Or the compositor making random guesses?
>
> > In the many-gamepads-per-seat model, I don't know how to set this up. The
> > first two players would have to have their own seats, but since the
> player
> > ID is in the gamepad, where do we get their player numbers from? And what
> > about players 3 and 4? Where do their gamepads go? They can go in seat 1,
> > but then how do you distinguish player 1 from players 3 and 4? And why
> does
> > player 2 have to manually focus the game?
>
> The gamepads go to whichever seat it is that's directing their focus.
> If their focus is entirely manually set, then they go in their own
> seat.  Simple, no? Or am I totally missing something, and do gamepads
> under Wayland typically not follow keyboard/mouse focii? (I don't mean
> in really contrived theoretical cases, I mean in the overwhelming
> majority of situations.)
>
> > (I don't believe this scenario is inherently exotic. Lemmings, Settlers,
> and
> > Hired Guns all did this back in the DOS and Amiga days, before OSs made
> it
> > harder to distinguish mice from each other.)
>
> Yes, that's the tradeoff you get for multitasking, and having the OS
> know where your mice are, rather than having to tell every single game
> where every single mouse is.
>
> > B) Imagine that, with the current trend of adding touchpads to gamepads,
> > Fruit Ninja releases a multi-player version. All players play on the same
> > "field", but it tracks their scores independently; whomever slashes the
> > fruit first gets the points. Let's imagine we have player 1 with a
> separate
> > touchpad, and players 2 and 3 with gamepads that have touchpads built in.
> >
> > In the seat == user model, it's obvious how this should be set up: seat 1
> > gets a touchpad, seat 2 gets a gamepad with the associated touchpad, and
> so
> > does seat 3.
> >
> > In the many-gamepads-per-seat model, the question rises again of where
> you
> > put the gamepads. Do they go in the same seat as the touchpad they're
> > attached to? Or do they all end up in seat 1? If they're all in seat 1,
> does
> > the player indicator on their controller match up with their actual
> player
> > number? And again, they all have to individually focus the game.
>
> Again, how does the focus get changed? Where are the gamepads deriving
> their focus from? Put them in that seat, and add a wl_touch object for
> every gamepad.
>
> >> If the focus can be changed independently, how does this happen?
> >
> > Focus can only be changed independently for "collaborative" seats.
> > "Collaborative mode" means "I want to interact with the desktop
> > independently." When collaborative mode is turned off, your focus is
> derived
> > from a seat which *is* in collaborative mode.
>
> OK, so what's the downside from having the gamepad embedded inside the
> seat it derives its focus from?
>
> >> If
> >> the keyboard/pointer seat switches focus to another game, do both
> >> seats switch, or does the other stay behind?
> >
> > Assuming seat 2 is following seat 1, they both switch.
> >
> >> If both switch - why are
> >> we complicating the focus model rather than just adding both to a
> >> seat?
> >
> > Because this problem doesn't start and end with gamepads. If we take the
> > multiple-gamepads-per-seat model to its logical conclusion, then to
> support
> > the scenarios above, you would also need multiple
> mice/keyboards/touchpads
> > per seat. And then how do you determine which devices belong to which
> players?
>
> No, we don't because we don't lose information by aggregating them.
> If you want to support multiple independent mice, then the expectation
> is that the mice can keep on going straight out of the window, and
> then change the focus.  So either it changes the focus for everyone
> (same seat), or for just it (separate seat).
>
> If you want relative mice, Kristian's relative work should probably be
> extended to provide one object per device, similar to wl_gamepad,
> since aggregating it becomes a lot less useful.
>
> But if it doesn't have any involvement or interaction with the core
> shell UI, then why is it a seat? A seat has a pretty singular meaning
> right now, and I'd rather not complicate and change that for all
> clients, and make them contort themselves in order to support a fairly
> niche usecase.
>
> Maybe I'm just being thick, but I honestly can't see the downsides of
> the multiple-gamepads-per-seat model.  One seat is currently
> restricted to one of each device, but that's only because we were able
> to aggregate those devices without losing information.  For gamepads,
> that's blatantly not true, but they don't have interactions with the
> shell and core UI - that's driven by the other devices in the seat, or
> by the shell's own policy - so the usual answer of 'put them in
> another seat' doesn't apply I think.
>

I think what Rick (and others) are trying to do is avoid protocol
complexity and complexity for the clients by making things just a little
more complex (not much if at all) server-side.  We have already broken the
idea of "seat == focus" because every seat has a keyboard focus and a
pointer focus which might not be the same.  The idea presented is to,
instead of "seat == focus" have "seat == user" or "seat == collection of
input devices".

>From the protocol perspective, there is nothing whatsoever that says that
any of the focus concepts have to derive from the pointer focus of the
seat.  For instance, nothing says that the seat's focus can't come from
touch.  Similarly, there's nothing preventing a seat's focus coming from
another seat.  I think "collaborative mode" is a bad way to think about
it.  Rather, the server groups input devices into seats and then focuses
those seats based on pointers or touch.  If there aren't enough pointers
for each seat to have one, then some of the seats become "slaves" to other
"master" seats.  However, this is entirely a server implementation detail,
the client does not need to know.  All the client knows are that there are
4 seats.

Why does it make things simpler?  Well, there are all sorts of interesting
use-cases out there:

1. Let's say that everyone has PS4 controllers with a keyboard attachment.
Now, we 4 gamepads, 5 mice, and 5 keyboards (one mouse and keyboard from
the desktop setup).  However, you probably don't want to spend time
navigating out-of-game with the PS4 controler.  With the above method, the
controller seats simply become slaves of the desktop seat.  This makes for
easier focus management for the user and the keyboards/mice/gamepads are
nicely grouped for the game.

2. You want to play a 4-person mario-style platform jumper with keyboard
controls.  Instead of trying to crowd everyone around the same keyboard,
you plug in 4 keyboards.  However, those cannot be aggregated because the
game needs to see them as distinct, but the user does need a way to focus
them all on the game.  Again, you set up 4 seats 3 of which are slaves to
the "master seat".

Really, it solves a lot of potential input issues (even with the current
objects) without breaking the protocol.  The DE can provide some sort of
GUI to the user to configure all this stuff.  That GUI may not mention
seats at all, but that doesn't really matter either.

Hope that makes more sense,
-- Jason Ekstrand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/wayland-devel/attachments/20130514/6f85de41/attachment-0001.html>