[PATCH wayland] protocol: define further the behavior of input on the presence of grabs

Thu Jun 11 18:24:16 PDT 2015

On Thu, Jun 11, 2015 at 02:33:34PM +0200, Carlos Garnacho wrote:
> On mié, 2015-06-10 at 08:20 -0700, Jasper St. Pierre wrote:
> > On Wed, Jun 10, 2015 at 4:50 AM, Carlos Garnacho <carlosg at gnome.org> 
> > wrote:
> > > On mar, 2015-06-09 at 11:03 -0700, Jasper St. Pierre wrote:
> > > > What is the value of clients having this information?
> > > 
> > > The same there is in having wl_touch.cancelled. See the cases in 
> > > the
> > > original email in the thread.
> > > 
> > > > I think we've
> > > > tried to hide grab semantics from the point of view of the client 
> > > > and
> > > > simply say "we can revoke input at any time, including in 
> > > > response to
> > > > a request you make", which gives us, the compositor, a lot more
> > > > leeway
> > > > with future possibilities.
> > > 
> > > Sure, I'm all for that. What I'd really want is having a way to 
> > > tell
> > > clients we do so. I'm totally fine with documenting the places 
> > > where
> > > this potentially happens on a per-case basis, even if the final 
> > > wording
> > > is "compositor dependent", same as for seat.release_input emission.
> > > 
> > > That said, toolkits/clients will surely expect some consistence 
> > > across
> > > compositors, so the leeway is relative.
> > > 
> > > > 
> > > > Grabs were a major pain point in X11 since they were 
> > > > overspecified,
> > > > so
> > > > we're trying to not fall into that same trap again.
> > > 
> > > I'm sure there is a lot of ground between "let's not overspecify" 
> > > and
> > > "let's go shopping".
> > > 
> > > It would be convenient first to identify what are the sore points 
> > > with
> > > X grabs. AFAICT, most of these come from grabs not being easily
> > > transferred, and the WM/screensaver/etc not being more of a client 
> > > to
> > > revoke/break grabs. On wayland the compositor is completely free to 
> > > do
> > > as it pleases, with and without this change I'm proposing.
> > 
> > Yeah, transferring grabs race-free, a lack of being able to override
> > or revoke grabs are the top two. But focus management + grabs in X11
> > is tricksy and sort of awkward: if I take a keyboard grab, key focus
> > can still navigate around as usual, we'll just get NotifyWhileGrabbed
> > as our detail.
> > 
> > > However, one thing that X did well is defining a consistent event
> > > delivery model when grabs were being taken (well, except for touch
> > > events...), so both the grabbing and the pre-grab windows are well
> > > aware of what's going on, I think one is due in wayland, at least 
> > > face
> > > to clients.
> > 
> > Did it? I don't know of any model that lets me know when a client has
> > taken a grab or ungrabs their existing grab. 
> 
> The same happens on wayland *between clients*, and that's a good thing.
> Sure, it wasn't convenient on X11 for a WM, as yet another client.
> 
> > The exception is that I
> > believe if I'm the key or pointer focus, I'll get a FocusOut / Leave
> > event with the "NotifyGrabbed" detail, and when the grab is dropped
> > (and I am still the key focus / pointer focus, which is not
> > guaranteed!), we'll get the reverse event with "NotifyUngrabbed".
> 
> Precisely, a focus out/leave event with NotifyGrabbed means "I may
> never see this device again", The second enter/focus in event may not
> happen because the pointer/keyboard focus went elsewhere mid-grab, but
> for all that matters, the client already forgot about the device(s), so
> is in a consistent state.
> 
> > 
> > And in X11, this is actually good, because such an event would be
> > racy: some other client might have taken a grab in such a time. And 
> > it
> > happens all the time because of passive grabs, including X11's own
> > implicit passive grab on the pointer.
> > Anyway, this model matches well with wl_keyboard.leave /
> > wl_pointer.leave being sent at the start of device grabs.
> > 
> > > > 
> > > > For instance, if the user is in a game that has a keyboard grab 
> > > > and
> > > > presses Alt-Tab to switch out, the client should just have its
> > > > keyboard grab revoked without having to have that as a 
> > > > possibility in
> > > > the protocol spec. Same thing with tray icon behavior, etc.
> > > 
> > > sure, in that case you'd still get wl_keyboard.leave and the client 
> > > can
> > > properly undo the key press / mods. But notice there is always a 
> > > need
> > > to know when to undo (eg. in your example above, the game might 
> > > have
> > > bound Alt to "release flares", if you first press Alt and then Tab, 
> > > and
> > > the client doesn't get the Alt key release, you don't want to leave
> > > that stuck when you focus back)
> > 
> > The client gets a wl_keyboard.leave. Is that not good enough?
> 
> Ok, we fixed keyboards. Keyboard is actually an easy usecase, they in
> the end have a binary state, even if for multiple keys, and not much
> retained state behind. Some observations though:
> 
> - This works here because in this alt-tab example focus is being taken
> elsewhere, say we're raising a notification area or anything else that
> only steals focus temporarily, how do we convey the app "you should
> still consider yourself focused nonetheless, and still should appear as
> such"?
> 
> - Do we take the same approach with wl_pointer, start telling "You must
> forget everything about this pointer on wl_pointer.leave events" and
> issue it mid-press if necessary? What about the legit places you may
> want enter/leave events while a button is pressed (eg. moving across
> subsurfaces)
> 
> - Let's say this is enough for keyboards and pointers. What do we do
> about interfaces with no "leave" semantics like wl_touch?
> wl_touch.cancelled is not too thinly grained, mostly thought out for
> compositor gestures taking over the whole device/capability. Issuing
> wl_touch.up to "cancel" individual touches, or touches happening in an
> specific surface is definitely not what we want here.
> 
> - Looking a bit forward to the future, we'll have tablets and
> buttonsets too, tablets will be specially funky as there's different
> interaction levels ("proximity" and pen in contact, pen buttons work on
> both states...), how do we behave orderly wrt the combination of those
> states?
> 
> It strikes me as simpler having a new event that will also work for
> future capabilities than cramming current events with vague semantics
> that toolkits would need to heavily adapt for.

If wl_touch.cancelled is too general (doesn't allow cancelling touches
one one of many surfaces owned by one client), I think it makes much
more sense to introduce a "wl_touch.cancelled_on_surface" which is what I
assume you want. I don't think it is simpler to have a catch-all event
that even is redundant for pointers and keyboards. For future buttonsets
and other input device types, it might not even have seat capability
enums, so it wouldn't catch those cases as well. It would also have be
more confusing semantics, since the catch-all-cancel event would still
need per-device-type specifications.

Jonas

> 
> > 
> 
> >  What use
> > cases does this new event solve?
> 
> I again recommend you to read http://lists.freedesktop.org/archives/way
> land-devel/2015-April/021407.html
> 
> In short, I see all sorts of problems with simultaneous interaction
> (multitouch alone, and combinations of any two device capabilities),
> where one happens to trigger a xdg_popup, dnd or another compositor
> grab. The state in which the former/grabbing surface are left is both
> unspecified and broken in implementations.
> 
> Cheers,
>   Carlos
> _______________________________________________
> wayland-devel mailing list
> wayland-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/wayland-devel