[RFC] Interface for injection of input events

[Pekka Paalanen]

Hi Peter,

more questions but not necessarily for you to answer - people who want
to make use of the proposed interface should ideally answer most of
these.

On Fri, 24 Mar 2017 15:04:57 +1000
Peter Hutterer <peter.hutterer at who-t.net> wrote:

> On Wed, Mar 22, 2017 at 01:59:43PM +0200, Pekka Paalanen wrote:
> > On Wed, 22 Mar 2017 12:23:46 +1000
> > Peter Hutterer <peter.hutterer at who-t.net> wrote:
> >   
> > > Hi all,
> > > 
> > > This is an RFC for a new interface to generate input events from arbitrary
> > > clients. Note that this is only a few days old, so **do not** assume this is
> > > anything more a thought experiment right now. This email is supposed to start a
> > > discussion and collection of the various points that need to be addressed.

> > > == Isolation of devices ==
> > > Compositors should create separate virtual input devices for each client so
> > > one client can't mess with the state of another one or even detect if
> > > there's another one active. Plus we get to re-use all the existing code that
> > > merge state from different (physical) devices. This just makes the actual
> > > device handling implementation trivial.
> > > 
> > > It gets a bit more difficult if we want to have this per-seat though. Seat
> > > is a wayland concept so we could fold that in, but then you're leaking the
> > > seat configuration. So I'm not 100% sure we even want this, or whether we
> > > should just make sure it can be retrofitted in the future.  
> > 
> > One possibility is to always create at least one new virtual seat
> > (wl_seat) for the emulated input devices. Woodotool clients would see
> > the virtual seat and possibly be allowed to create and destroy more
> > virtual seats at will. This would not leak or interfere with the real
> > seats.
> > 
> > At the same time it would also exclude use cases where one wants to
> > hook into the real seat, e.g. a program that provides keyboard events
> > but relies on the real seat for focus handling.
> > 
> > There can be multiple real seats. How would one pick which real seat to
> > integrate with?
> > 
> > Hmm, wait, what do mean by "leaking the seat configuration"? Don't we
> > expose exactly that straight through Wayland? What's the issue you see?  
> 
> The way seats are exposed over wayland is a bit different than over DBus. A
> wayland client may never see another seat if we restrict it in the
> compositor. Could be done over DBus but needs some extra handling - again
> that goes hand-in-hand with the authentication bits.
> 
> Having a separate seat for a virtual device means we have to worry about
> focus handling. The compositor could always trail the real seat for the
> virtual seat (which would make different xkb layouts trivial). So, an
> interesting idea but I'm not sure how well clients are designed for multiple
> keyboard foci.

That may be true about keyboard foci, but we already have it in Wayland
protocol.

As for "trail the real seat", the question still remains: which of the
possibly multiple real seats? :-)

Do you want to architect the interface such that the window system
protocol used does not have to have multi-seat (multi-focus) features?
I can understand that desire. Then some thought needs to go into how to
support both multi-focus and single-focus window systems.

Would it be enough to just always tie all virtual input devices to
"the first seat" on multi-focus systems?

Btw. focus handling could be just punted, like it is done on real input
devices: real input devices cannot say "this goes to that particular
window", so we could just make virtual input devices behave exactly the
same while still having a separate virtual seat.

But if the client (event producer) of the virtual input device
interface has no idea and cannot directly assign the foci, is it still
useful for the use cases it is being designed for?

> > > == Input coordinate filtering ==
> > > One use-case that was mentioned to me was effectively "intercept button X
> > > and send key events 'abc' instead". This is not possible with the current
> > > proposal and I'm not sure it can be without going overboard with
> > > specifications. It *may* be possible to provide some global hotkey hooks I
> > > have not come up with a good solution to that.  
> > 
> > A global hotkey system has been discussed in the past. IIRC one of the
> > better ideas was something along the lines of (Wayland) clients
> > registering "actions" with default "input event" bindings but keeping
> > the control in the compositor. The binding would not work by default for
> > most input events, except maybe for some whitelisted things like
> > Play/Pause keys, and would require the user to accept the binding. The
> > user would also have the opportunity to bind the actions to any other
> > input events he wishes.
> > 
> > Again such a "grant permissions" scheme must rely on some form of
> > persistent application identification, which quickly moves us towards
> > containers again if we want to be resistant towards clients
> > deliberately lying what they are.
> > 
> > However, IMHO such button/key etc. remapping is something that should
> > be done completely in the compositor (if not possible in keymaps) to
> > make it reliable and fast, unless perhaps when the original input
> > device is not one that the compositor is already using, like a joystick
> > or a gamepad. Or a smart phone acting as a touchpad/keyboard for remote
> > control of a computer.  
> 
> Right, I generally agree with that idea, the question here is whether this
> is something we should look into providing this over dbus (i.e. as part of
> an interface as discussed here) instead of wayland. Wayland has some
> advantages though like focus handling, better context of whether the
> application is displayed atm, etc.

Sorry, I don't understand how using Wayland instead of DBus would help
anything. The program generating virtual input events would not gain
any of that by simply using Wayland, would it? It would never know
about foci on other clients' surfaces without adding new interfaces,
which would be essentially the same if it used DBus.

Therefore I would claim that using Wayland has no advantages, aside
perhaps from naturally exposing the wl_seats. Nothing else readily has
global identifiers on Wayland.


Thanks,
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20170324/8701bad5/attachment.sig>