Compositor grabs (was: Re: [PATCH] protocol: Add DnD actions)

[Jonas Ådahl]

On Fri, Apr 17, 2015 at 02:16:41PM +0200, Carlos Garnacho wrote:
> Hey Jonas,
> 
> This is drifting a bit off the topic of the original thread, better to
> spin this off. I'll reply to the DnD bits in another email.

Hey, sorry for the delay. Some comments/questions below.

> 
> On Fri, Apr 17, 2015 at 9:50 AM, Jonas Ådahl <jadahl at gmail.com> wrote:
> > On Thu, Apr 16, 2015 at 12:55:31PM +0200, Carlos Garnacho wrote:
> >> Hey Jonas,
> >>
> >> On Thu, Apr 16, 2015 at 10:15 AM, Jonas Ådahl <jadahl at gmail.com> wrote:
> >>
> >> <snip>
> >>
> >> >
> >> > I'd have to agree on that it doesn't seem like the best thing to let the
> >> > compositor choose the preferred action. Having it apply compositor
> >> > specific policy given what the keyboard state or similar will probably
> >> > never work out very well, given that for example what modifier state
> >> > means what type of action is very application dependent.
> >> >
> >> > On the other hand, I'm not sure we can currently rely on either side
> >> > having keyboard focus during the drag. In weston the source will have the
> >> > focus because starting the drag was done with a click which gave the
> >> > surface keyboard focus implicitly, but what'd happen if the compositor
> >> > has keyboard-focus-follows-mouse? We could probably say that drag implies
> >> > an implicit grab on another device on the same seat to enforce no
> >> > changing of keyboard focus, but not sure that is better.
> >>
> >> In gtk+/gnome we currently have the following keybindings active during DnD:
> >>
> >> - Cursor keys move the drag point, modifiers affect speed
> >> - Esc key cancels drag
> >> - Modifiers alone pick an action from the offered list
> >>
> >> So ok, the latter is dubious to punt to compositors, but there's
> >> basically no other choice with the 2 first ones.
> >>
> >> More generally, I have the opinion that compositors grabs should
> >> behave all consistently, as in:
> >>
> >> - Ensuring clients reset all input state (we eg. don't cancel ongoing
> >> touches when xdg_popup/dnd/... grabs kick in)
> >
> > What does "client reset all input state" mean? What state can a client
> > reset?
> 
> Let's expand on that example, maybe far-streched, but certainly possible:
> - I'm manipulating a client window with 2 fingers on the touchscreen
> (say zooming an image)
> - Any other interaction on the client makes it pop up an xdg_popup
> (say a third touch, a key combo, or the pointer)
> - Q: what happens with the two first touches?
> 
> Ideally, these touches should be "cancelled" on the first surface
> (wl_touch.cancelled seems to be per client, not per-surface though),
> and stay non-reactive within the grab, so they wouldn't trigger
> anything unintended (they're implicitly grabbed on another surface
> after all)

It seems rather unspecified in the protocol, but AFAIK at least in
weston only one surface may have any touch points active at any given
time. Mutter seems to do the opposite, i.e. any number of surfaces can
have "touch focus". I'm don't know what is intentional here, but
assuming single touch focus is, wl_touch.cancelled being per client is
not a problem when an explicit grab is initiated on another surface,
since we'd have to cancel the touches of the previously focused surface
(which is all on that client), and then start them on the new one (hmm,
should the popup get new 'down' events? I assume it must, since one
should not have to raise the finger and then touch again to interact
with the popup).

Or, the other way, where there is no such thing as single touch focus.
I.e. that multiple surfaces can have touch points at the same time, then
we'd need changing the protocol to support what you describe (as as you
say, wl_touch.cancelled is per client).

Anyway, either we need to properly clarify that in wl_touch I think, and
it'd be helpful to know if this was considered when designing the
protocol.

> 
> Currently, on the weston code, focusing a bit on the all-touch case,
> it actually happens the worst that could happen, the xdg_popup touch
> grab redirects already started touch sequences to the grabbing surface
> right away, and the original surface will be deaf to them of a sudden,
> leading to inconsistent state on both the original and the grabbing
> surface wrt those touch sequences. The DnD touch grab doesn't fare
> much better, it will ignore other touches than the one starting the
> drag, so the pre-grab touches would effectively go nowhere, and AFAICS
> similar issues arise with pointer grabs vs touch.

Yes this seems wrong.

> 
> With keyboards, it happens likewise, if compositors are to possibly
> consume events there, focus should move out of the previous surface.
> IMO, any grabbing model that does not redirect all input, nor ensures
> a coherent state in clients is calling for trouble...
> 
> In the X11 world, this would roughly be a Virtual Core
> Pointer+Keyboard grab (not that touch and active grabs are trouble
> free in X11, but...), GTK+ for example does grab both devices on every
> of those grabbing places wayland/xdg protocols are trying to cater for
> (I've even pondered about adding a gdk_device_grab_pair() for years).
> 
> I think some consistent model should be devised here, and embedded
> into the protocol (docs).

I agree that some consistency would be good. How do you suggest we do so?
Should we make it globally enforced semantics, meaning formally
specifying what an explicit grab is, or should it be per protocol that
uses them? I wonder if there will be explicit grab types that require
special casing. For example the discussed pointer lock/confine takes
something similar to an explicit grab, and it seems reasonable to make
this grab also take the keyboard focus, should it also take the touch
focus?

> 
> >
> >> - Ensuring the grab affects the routing of all devices/events, and
> >> that no client gets partial streams
> >
> > You mean that pointer grab should implicitly grab the touch and keyboard
> > device as well? What do you mean with partial stream?
> 
> See the example above with surfaces receiving not properly
> started/finalized series of event for touch sequences, or the
> suggestions in the DnD thread about having drag source/dest handle
> keyboard events, while the compositor is certainly interested in
> handling/consuming some keyboard events too.

Well, the compositor personally doesn't care about grabs. It can do
whatever it wants. But if we want the grab consistency you are talking
about, do you mean that the source should loose keyboard focus when the
grab is initiated? Only for Xwayland surfaces the pointer focus will be
left intact, regular ones will loose it.


Jonas

> 
> Cheers,
>   Carlos