[PATCH weston 1/3] Introduce pointer lock interface

[Pekka Paalanen]

On Fri, 19 Sep 2014 20:20:53 +0200
Jonas Ådahl <jadahl at gmail.com> wrote:

> On Fri, Sep 19, 2014 at 11:00:21AM -0600, Jasper St. Pierre wrote:
> > On Fri, Sep 19, 2014 at 2:54 AM, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> > 
> > > On Fri, 19 Sep 2014 08:33:13 +0200
> > > Jonas Ådahl <jadahl at gmail.com> wrote:
> > >
> > > > On Thu, Sep 18, 2014 at 10:43:47AM -0700, Jason Ekstrand wrote:
> > > > > On Thu, Sep 18, 2014 at 2:26 AM, Pekka Paalanen <ppaalanen at gmail.com>
> > > wrote:
> > > > >
> > > > > > On Wed, 17 Sep 2014 22:35:40 +0200
> > > > > > Jonas Ådahl <jadahl at gmail.com> wrote:
> > > > > >

> > > You could create a wl_pointer, if you do it from interface whose
> > > ancestor is wl_seat, or directly a request in wl_seat. But it does have
> > > downsides like not being able to let the extension mature in Weston
> > > before moving it to Wayland core.
> 
> I think its a good practice to have a isolateable kind of way of
> extending protocols (ala wl_relative_pointer, wl_scaler, etc) so that
> core protocol objects don't grow too large with non-removable request
> and events.

Indeed.

> Maybe we need to define "modes of operation" that would be similar to
> "roles" (e.g. xdg_surface), so that wl_relative_pointer changes the
> "mode" of wl_pointer to wl_relative_pointer. This would allow us to add
> a wl_pointer_lock.confine_pointer that doesn't make sense to enable at
> the same time as the pointer is in relative motion mode.

Roles are not really modes. Some roles do affect the wl_surface
state somewhat, but they do not seriously alter the behaviour of
wl_surface requests or events.

Having relative vs. absolute modes on a wl_pointer would, though.

I'd rather avoid modes where we can, especially on protocol
objects. The underlying thing may have modes where it chooses which
protocol object it targets, which is a bit less confusing than
switching what a protocol object does.

> > > > non-sensible requests (set_cursor) on the additional wl_pointer, and
> > > > potentially more as wl_pointer is extended. The alternatives that has
> > > > come up in IRC discussions and backlog reading are:
> > > >
> > > >  1) Create a wl_relative_pointer that would replace wl_pointer, i.e.
> > > >     have its own button, axis, etc events. This would solve all problems
> > > >     related to versioning and wl_pointer being extended, but brings a
> > > >     bunch of other problems for example we'd have to extend
> > > >     wl_relative_pointer each time wl_pointer is extended to keep up, as
> > > >     well as the issue where a wl_pointer is used in a interface or
> > > >     request.
> > >
> > > Either you use wl_pointer and get extensions automatically even if you
> > > don't want to, or you use a new interface that you have to extend
> > > explicitly if you want the same as in wl_pointer. Two sides of a coin.
> > >
> > > As concluded in IRC, wl_pointer is not used as argument in any messages.
> 
> But it is by wl_pointer_lock.lock_pointer, and who says it'll be the only
> place its used as an argument?

I think wl_seat might be a better argument. After all, all
wl_pointer objects created from the same wl_seat in the same way
will be identical. I think we should keep it that way, too.

> > > >  3) Extend wl_pointer similar to how wl_viewport extends wl_surface with
> > > >     a wl_relative_pointer (different from in (1)). Such a
> > > >     wl_relative_pointer would, to begin with, only have one request
> > > >     (release) and one event (motion). It wouldn't change anything to the
> > > >     wl_pointer object, it'd still receive button, axis etc events, but
> > > >     motion events are turned into relative motion events via
> > > >     wl_relative_pointer, not affecting the position of the cursor, i.e.
> > > >     not resulting in wl_pointer.motion events.
> > >
> > > This is an interesting idea. I wonder how it works if the client has
> > > two parts (say, libraries or toolkits with their own input processing),
> > > therefore has multiple wl_pointer objects for the same thing, and only
> > > few of them have the pointer_lock interface associated.
> 
> Hmm. Maybe we need to lock pointer given a seat instead. It'd change the
> "mode" of the underlying pointer to be locked in its position, i.e. not
> send any wl_pointer.motion events. Locking could then be ref counted, so
> that as long as more than one wl_relative_pointer is alive, it's in
> relative mode. Maybe thats unnecessarily complex though.

We are distinguishing "wl_relative_pointer exists" from
"pointer-lock is active", right?

A question is: how does the client know when pointer-lock is
active? I think we'd need something like
wl_relative_pointer.enter/leave.

> > > Or what if an app manually starts pointer-lock because the toolkit does
> > > not support it? This is not a use case to design for at all, but if it
> > > happens to work, it might be a nice bonus.
> 
> If we make it so that locking simply locks the cursor at its position,
> it'd will just work, as the pointer cursor wouldn't move. No unexpected
> wierd wl_pointer.motion events.

You also need to account for button, axis etc. events, even if the
pointer appears to stay put.

> What would a set_cursor_position argument do? It sounds like that could
> be more or less used to implement actual warping by just locking, set
> position then unlocking. It'd probably be more useful to have a
> set_cursor_position request in wl_relative_pointer that can be used when
> implementing pointer warping in Xwayland and then possibly even SDL and
> GLFW. Hmm.

Like I commented in IRC, doing this for warping may not work. We
haven't figured out the usual conditions where the pointer-lock
would actually activate. We probably even leave that up to
compositors, which means a client cannot reliably "just lock" but
may need some further user action.

If a compositor uses focus-follow-mouse, it would be very annoying
just moving accross a window with existing wl_pointer_lock to
activate the lock, so such a compositor would likely require a
click or raise on the window.

> > > Though, I don't see anything wrong with requiring the client to
> > > explicitly unset the cursor when pointer-lock activates. Or are there
> > > any room for races where the cursor might flicker?
> 
> Well, except for the buffer queue extension, is there any way for two
> surfaces to synchronize their rendering (assuming they are not not in a
> subsurface tree)?

No.

But that is not what I meant. I meant a case where some state on
the cursor surface or wl_pointer causes it to become visible with
one image and some location, and only afterwards the client has the
opportunity to fix the image (and location) to match reality. That
kind of flicker.

Note, that if we use the standard wl_pointer semantics of enter
event, there cannot be flicker, because the client needs to repond
by setting the cursor image. That's another argument for causing
the normal wl_pointer to emit leave when pointer-lock activates.

> > > There are at least to different basic use cases:
> > > - Quake, i.e. FPS-games with mouse-look
> > > - a strategy game, which still uses a cursor, but implements special
> > >   behaviour when running against window edges like scrolling the view,
> > >   likely with the pointer confined to the window
...
> > > A good question is, do we want to support the latter with
> > > pointer-lock, or should it be another extension?
> 
> I imagine we could have a wl_pointer_lock.confine_pointer, if we want to
> support such a use case directly in Wayland.

That is not a reason here. If a client runs on relative events, it
can trivially implement confining in any way it wants, without
compositor's help. E.g. when trying to move over the window edge,
do not move the cursor but move the window content instead (like RTS
map).

The big question here is the acceleration or not.

> > Also, the latter probably wants the same motion acceleration algorithm
> > > as the desktop has, but the former probably not.
> > >
> 
> I can imagine that the second use case may very well be implemented both
> with confinement and relative pointer events and as such relative
> pointer events would need support both non-accelerated and accelerated
> modes.

IMHO that raises the question: should be have separate interfaces,
or at least separate requests for asking raw relative motion vs.
desktop-like relative (i.e. accelerated) motion?

Or should wl_relative_pointer maybe deliver both at the same time?

Maybe have separate requests for creating the wl_relative_pointer...


Thanks,
pq