RFC: libei - emulated input in Wayland compositors

[Vlad Zahorodnii]

Howdy,

On 7/31/20 8:13 AM, Peter Hutterer wrote:
> I've been working on a new approach for allowing emulated input devices in
> Wayland. Or in short - how can we make xdotool and synergy work? And
> eventually replace them.

++, it would be nice to have something that is supported by all major
compositors + working synergy on Wayland. :-)

> The proposal I have is a library for Emulated Input, in short libei.
>   https://gitlab.freedesktop.org/whot/libei/
> 
> libei has two parts, the client side (libei) for applications and
> a server side (libeis) for the compositor. The two libraries communicate
> with each other (how? doesn't matter, it's an implementation detail) to
> negotiate input devices.
> 
> The process is roughly:
> - the libei client connects and says "I am org.freedesktop.SomeApplication
>   and I want a pointer and a keyboard device"
> - the libeis server says "ok, you can have a pointer device and a keyboard
>   device"
> - the libei client says 'move the pointer by 1/1', etc. and the server does
>   just that. or not, depending on context.
> 
> There are more details, see the README in the repo and the libei.h and
> libeis.h header files that describe the API.
> 
> The sticking point here is: emulated input comes via a separate channel.
> The server a) knows it's emulated input, b) knows who it is coming from and
> c) has complete control over the input.
> 
> a) is interesting because you can differ between the events internally. The
> API right now is very similar to libinput's events so integrating it into a
> compositor should be trivial.
> 
> b) is somewhat handwavy if an application runs outside a sandbox - any
> information will be unreliable. Flatpak gives you an app-id though and
> with that we can (eventually) do things like storing the allow/deny
> decisions of the user in the portal implementation.
> 
> c) allows you to e.g. suspend the client when convenient or just ignore
> certain sequences altogether. The two made-up examples are: suspend EI
> during a password prompt, or allow EI from the software yubikey *only*
> during a password prompt.

We don't want any application be able to inject input events. How should
a program such as synergy authenticate itself? or is it completely up to
the compositor to decide what the authentication process looks like?

Cheers,
Vlad

> Now, the next question is: how do they *start* talking to each other?
> libei provides multiple backends for the initial connection negotiation. My
> goal is to have this work with flatpak portals so an application running
> within the sandbox can be restricted accordingly. Alternatives to this could
> be public DBus interfaces, simple fd passing or (as is implemented right
> now) a named unix socket.
> 
> The aim is that a client can simply iterate through all of the options until
> finds a connection. Once that's found, the actual code for emulating input is
> always the same so it's trivial to implement a client that works on any
> compositor that supports some backend of libeis.
> The server part only needs to care about the negotiation mechanisms it
> allows, i.e. GNOME will only have dbus/portal, sway will only have... dunno,
> fd exchange maybe?
> 
> Next: because we have a separate channel for emulated input we can hook up
> XTEST to use libei to talk to a compositor. I have a PoC implementation for
> weston and Xwayland:
>   https://gitlab.freedesktop.org/whot/weston/-/commits/wip/eis
>   https://gitlab.freedesktop.org/whot/xserver/-/commits/wip/xwayland-eis
> With that xdotool can move the pointer. Note this is truly the most minimal
> code just to illustrate the point but you can fill in the blanks and do
> things like the compositor preventing XTEST or not, etc.
> 
> This is all in very early stages with very little error checking so things
> will probably crash or disconnect unexpectedly. I've tried to document the
> API to make the intentions clear but there are still some very handwavy
> bits.
> 
> Do let me know if you have any questions or suggestions please though.
> 
> Cheers,
>   Peter
> _______________________________________________
> wayland-devel mailing list
> wayland-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/wayland-devel
>