Gamepad focus model (Re: Input and games.)

[Rick Yorgason]

Jason Ekstrand <jason at ...> writes:
>> Scenario 2) Two users are using a multi-user display server. Each user
>> has a keyboard, mouse, and gamepad. User 2 has to set up their wl_seat
>> using some
>> configuration window built into the display server, but once that's done
>> each user can jump in and out of a game as they see fit, assuming the
>> game was written to iterate over wl_seats properly. Neat!
>>
>> Scenario 3) Two users are using gamepads with built-in touch screens
>> (like the PS4 and OUYA controllers). The players don't care about the
>> ability to have their own window focus, but the game wants to associate
>> each touchpad with its gamepad. Player 2 must set up their own wl_seat
>> in the display server, and must focus the game separately using the
>> touchpad. Not ideal.
>>
>> You might say, "yeah, it's a bit of extra effort to set up player 2, but
>> the cool thing is that player 2 can jump in and out of the game as they
>> see fit!" Only that's not necessarily true either. Since many games
>> steal the cursor, and player 2 has no keyboard, player 2 can jump into
>> the game (using the touchpad on the controller) but can't jump out.

> I think this is where a judicious use of the "home" button could come in
> very handy.  As long as the game is nice and releases the mouse on
> "home", or the compositor takes over or something, the problem is solved.

Well, *a* problem is solved, but if you read carefully you'll see that's not
actually the problem the user was trying to solve. You're still saying
"Yeah, I know it's a pain to set up your controller, but look at this neat
thing you can do that you never really cared about!"

>> Scenario 4) In this scenario we imagine that wl_seat has grown support
>> for wl_headset. Player 2 has an Xbox 360 controller with a headset
>> plugged in. To support this, player 2 must go through the display
>> server's wl_seat configuration to set up their own seat. Like in
>> scenario 3, this is less than ideal. And how does player 2 focus the
>> game without a keyboard or mouse?

> I don't think we should restrict this to "default".  Effectively, we
> would need a way for a seat to have a "parent" seat from which it derives
> its focus.  This may get to be a mess especially from a user-interface
> perspective.  Perhaps you could have a "automatically parent to whatever
> my default is" paradigm.  However, for more interesting configs, it get a
> lot more confusing.  See below.

Yes, a parent seat is definitely more general. I was originally thinking of
something like that, but a use for it didn't immediately come to mind. The
two pairs of players scenario is interesting.

> Having the two controllers paired doesn't solve 3.  There are a lot of UI
> problems with having two pointers running around the screen that share a
> focus.  Let's say they're one of those crazy users that like sloppy-
> focus.  What happens when the two cursors are on different windows?  Does
> the primary cursor over-ride?  Does the secondary cursor work in windows
> that aren't focused?  Even if you have click-to-focus, you still have
> problems with the two cursors fighting.  What happens if they both try to
> do a drag-and-drop?  The only solution to this is probably to make the
> secondary pointer inert in windows that aren't focussed.  That said, I'm
> not sure if that 100% fixes it either.

I was thinking that a desktop environment would probably only have one
cursor. In other words, if any wl_seats are sharing focus, most apps would
treat them as aggregate, and it would only be more specialized apps (like
games) that might want to separate them.

> To tie up my comments, I think this can get far too complicated fast.  I
> think it's better to have a seat correspond to a focus and allow for the
> fact that we may have more than one person to a seat.  It's not 100%
> ideal, but it's a lot simpler than some strange tree of seats.  Besides,
> if some compositor wants to do a strange tree of seats, there's nothing
> in pq's proposed protocol that would prevent that and games shouldn't
> care.

pq's proposed protocol is certainly "good enough". My two concerns are:

1) Does this make it a pain to give players more than one IO device, like
gamepad+touchscreen or gamepad+headset.
2) If multiple wl_seats are only used in exotic setups, then programmers are
unlikely to discover that they're not enumerating them properly.

Both of these concerns are somewhat exotic, and I'm personally unlikely to
run into these problems, so I'm not going to lose any sleep if they're not
addressed. It just seems like wl_seats would be a perfect abstraction for
multiple players if only they didn't have to be so tightly tied to focus.

-Rick-