[PATCH weston 1/3] Introduce pointer lock interface

[Pekka Paalanen]

On Mon, 22 Sep 2014 18:36:23 -0700
Jason Ekstrand <jason at jlekstrand.net> wrote:

> On Mon, Sep 22, 2014 at 12:52 PM, Jonas Ådahl <jadahl at gmail.com> wrote:
> 
> > On Mon, Sep 22, 2014 at 11:31:27AM -0700, Jason Ekstrand wrote:
> > > On Sun, Sep 21, 2014 at 11:43 PM, Pekka Paalanen <ppaalanen at gmail.com>
> > > wrote:
> > >
> > > > On Sat, 20 Sep 2014 11:43:22 -0700
> > > > Jason Ekstrand <jason at jlekstrand.net> wrote:
> > > >
> > > > > On Sat, Sep 20, 2014 at 1:29 AM, Pekka Paalanen <ppaalanen at gmail.com

> > > > > > A question is: how does the client know when pointer-lock is
> > > > > > active? I think we'd need something like
> > > > > > wl_relative_pointer.enter/leave.
> > > > > >
> > > > >
> > > > > I'm still a fan of destroying relative pointers when not in use.
> > It's a
> > > > > bit more protocol churn, but it makes it clear when they are/are not
> > > > > active.  Maybe enter/leave will work, but I'm not convinced it's the
> > > > right
> > > > > way forward.  I'll have to give that a bit more thought before I
> > could
> > > > give
> > > > > a real solid argument though.
> > > >
> > > > Yes, of course. It's more a matter of how does the client know to
> > > > destroy the wl_relative_pointer, when the compositor chooses to
> > > > deactivate the pointer-lock. If there is a leave event for that, good.
> > > >
> > > > I think with this, it could actually make sense what you said above: no
> > > > normal wl_pointer events as long as any wl_relative_pointer exists.
> > > > This would strongly push app devs to destroy the wl_relative_pointer at
> > > > the right time, I believe.
> > > >
> > > > And then we need a backup plan for clients that do not destroy it, for
> > > > protocol definition completeness.
> > > >
> > >
> > > If the server sends a "leave" it becomes inert and the server assumes the
> > > client will destroy it.  If the client doesn't then you leak an object
> > but
> > > there's no real harm.
> >
> > So if a lock will be made invalid if it was deactivated when should the
> > client re-request the pointer lock? On activation could cause unwanted
> > pointer locking when the focus model is focus-follow-mouse (or actually
> > activation-follow-mouse which would be the equivalent I suppose) or does
> > the client need to keep a state of how a surface was activated in order
> > to postpone the actual locking?

As a first naive thought, clients that use pointer-lock would implement
some action to grab it, e.g. you have to click on the 3D-view in a FPS
game or something.

In activation-follow-mouse, I would want that, but you can argue that
also the compositor/shell can require it before activating an existing
pointer lock. However, then it becomes a problem with what to do with
button being down when pointer-lock activates. I wouldn't want the
character in FPS to start wasting ammo. Or maybe it would activate on
click, i.e. down-up.

Another case: say the shell uses click-to-activate, the game window is
inactive, and I want to move it. I do button-down on title bar and
intend to drag. It would not be good to activate pointer-lock at that
point. Client knows where the button-down happens, the compositor does
not. Again heuristics in the compositor could make it work, I believe,
but I don't really like relying on heuristics from the start in design.

Depending on how we design this, we could require a valid input serial
with creating a wl_relative_pointer, otherwise the compositor would
just immediately reply with wl_relative_pointer.leave. On first
thought, I would favour this, but I'm not sure how it would turn out.
This would be similar to how e.g. popup grabs are started.

Oh btw. yet another use case: say, an app has a 3D view as a (small)
part of its GUI, where you can drag to rotate the 3D-model. There the
app would attempt pointer-lock on button-down, and release it on button
up. That means we need to cater for buttons already down while
activating, I suppose that would just work. The converse is a bit
trickier if a compositor decides to break the pointer-lock while some
buttons are down: wl_pointer.enter does not carry a list of buttons
down!

> > Anyhow, given that there are enter/leave events sent to the
> > wl_relative_pointer object, what is left unclear regarding if a pointer
> > is or is not locked? Whats the benefit of invalidating deactivated
> > locks, and is it worth that the compositor will have less say about
> > relocking and rely on the client to know when its appropriate to do so?

If we go with invalidate, then:
- When a client creates a wl_relative_pointer, it will simply assume
  the pointer-lock is active.
- If the compositor disagrees, it replies with 'leave'. Just like with
  popups.
- If the compositor wants to break the pointer-lock, it sends 'leave'.

And when to create wl_relative_pointer I discussed above. I think the
client is in a better position to know when it wants a pointer-lock than
the compositor.

In this case, there might not be need for 'enter' event, and 'leave'
might be better called something else, because usually enter and leave
go in pairs, and can re-occur, i.e. do not make the object inert.

> The following is starting to get dangerously close to opinion that can very
> quickly head towards bikesheding.  We will need to get an real-world
> implementation or two before we can really know.  That said, I'm going to
> throw it out there anyway.
> 
> I really think that we want to always have the client ask for the lock.
> This means that if the compositor has to break the lock, the client doesn't
> get a pointer lock again until it asks for it again.  This leads very
> naturally to the client simply destroying the pointer lock object as soon
> as it gets the leave event.
> 
> Why do we want the client to have to ask?  Suppose we don't and that the
> client automatically gets the pointer back when they focus the window.  How
> does the compositor know if the client still wants it locked?  How does the
> client know where the pointer entered the window?  If you click on the
> window will the cursor just immediately jump back to where the locked
> version was left?  What happens if you have a portion of the window that
> has a lock and the first thing you click on in the client is a button?
> Since the absolute motion is deactivated, the client has no way to know
> where you clicked.  In other words, the cursor entering a surface is
> *always* absolute motion and it should be reported as such to the client.
> 
> As I said above, maybe we'll get a demo up and running that proves this
> wrong.  However, I think it's a good starting guess.

Yes, agreed 100%.

If I understood right, this would be slightly different to what was
discussed a little earlier: absolute motion is inhibited only if
pointer-lock is active, not when wl_relative_pointer simply exists
because it may be inert.

> > > > > > > I can imagine that the second use case may very well be
> > implemented
> > > > both
> > > > > > > with confinement and relative pointer events and as such relative
> > > > > > > pointer events would need support both non-accelerated and
> > > > accelerated
> > > > > > > modes.
> > > > > >
> > > > > > IMHO that raises the question: should be have separate interfaces,
> > > > > > or at least separate requests for asking raw relative motion vs.
> > > > > > desktop-like relative (i.e. accelerated) motion?
> > > > > >
> > > > > > Or should wl_relative_pointer maybe deliver both at the same time?
> > > > > >
> > > > > > Maybe have separate requests for creating the
> > wl_relative_pointer...
> > > > > >
> > > > >
> > > > > Both of those seem reasonable to me.
> > > >
> > > > I would suggest we go with a separate request with "raw" input data.
> > > > This way we can easily add it later. I suspect it may also need
> > > > libinput additions. We don't even have a good definition of "raw" yet,
> > > > either, AFAIK. Going with just the normal (accelerated) motion for now
> > > > should be easier to try out.
> > > >
> > >
> > > fine with me
> >
> > I think a separate wl_relative_pointer is overkill, but instead of
> > separate requests, could we not just have an "option" bitmask? Because
> > there really isn't much difference between a lock_pointer and
> > lock_pointer_noaccel. Haven't thought of any options more than "noaccel"
> > though, and maybe its a bad idea add more. Hmm.

I only meant to use wl_relative_pointer for both, just have two factory
requests. Having two factory requests is not a problem as long as they
share an ancestor in the interface hierarchy, where the global is the
root.

But...

> > Another thing that I thought of is that relative motions may not
> > necessarily be enough for confined absolute motion events, and that is
> > because of transformations. If we want to allow a client to *only*
> > confine the cursor, but still have it behave just like a regular cursor,
> > then surface transformations must be taken into account. While it could
> > be relevant to put such a thing in wl_pointer_lock, I guess it's better
> > to focus on only relative motion events first.
> >
> 
> That's a good point, but like you said, let's focus on relative first.

...that is a very good point. And one more question with confined
pointer is how to express how much the pointer is trying to move
outside the confinement, so that the app can use that to e.g. move a
map. Indeed, let's just think of the relative case first.

However, if the relative case is the FPS game case, then we would
probably want noaccel with it. Of course do not need that for the first
experiments, but we should start thinking what the "raw" input events
would be. I wonder if there is any way, based e.g. on the acceleration
settings, to establish a common scale for motion, so that hardware with
considerably different scale of motion would still be reported in a
roughly uniform scale to apps. Maybe libinput already has this?


Thanks,
pq