RFC: libei - emulated input in Wayland compositors

[Jonas Ådahl]

On Mon, Aug 03, 2020 at 02:57:11PM +1000, Peter Hutterer wrote:
> On Sat, Aug 01, 2020 at 01:42:16PM +0200, Roman Gilg wrote:
> > >
> > > 1) It exports a set of APIs under org.freedesktop.portal.* that all
> > > sandboxed applications can access.
> > >
> > > In contrast to explicitly allowed APIs (i.e. build time configured list
> > > of API to be exposed directly to inside an application sandbox by
> > > default), a portal APIs allows for applications to dynamically request
> > > access to privileged functionality, for example access to arbitrary
> > > file system locations, cameras, geo location, or screen casting.
> > 
> > That's quite a long sentence. Let me dissect it a bit. What kind of
> > API are we talking about? The xdg-portal-desktop one? Who defines the
> > "build time configured list of API"? Are we talking about the "build"
> > of the Flatpak? Is it the Manifest's "finish-args" as described this
> > way in the Flatpak documentation? [1]
> 
> sort-of, there's a confusion of terms. "API" stands primarily for a DBus API
> but you'll probably think of X11 and Wayland as sockets. for the purpose
> here you can consider them as APIs as well.
> 
> When you build a flatpak application, you specify what that application can
> access outside the sandbox, that's your [1] link. by default, a sandboxed
> application does not have X11 or Wayland access.
> 
> you can allow access to those or any API at build time, e.g.
> --talk-name=org.freedesktop.EI would allow the application to talk to a EIS
> server directly without the portal in between. These permissions are
> compiled in and listed by the flatpak itself - they're not dynamic like the
> portal.
> 
> Problem with any build-time permissions is the portal has no control over
> it. This particularly shows the issue with sockets which are a complete
> side-channel. A DBus API will be called every time
> it's needed - that allows for a gatekeeper in between. A socket, once
> established, is largely outside the control of such a gatekeeper. XTEST for
> example floats along on the rest of X, you cannot allow an application to
> use X for display but disallow XTEST in a portal, support for this would
> have to be implemented in X itself. The same would be true for a Wayland
> virtual input protocol.
> 
> EI is (will be) a combination of API + socket, similar to screencasting -
> negotiation is handled by the portal but once the connection is established
> the data is outside the portal's control. We could make the communication
> fully dbus-y so we can filter at any point but I don't think that would
> provide a lot of benefit.
> 
> The socket approach here is acceptable because it's very *specific* for what
> it does. It only does emulated input, so giving permission to that implies
> you want that to happen. Giving permission for X (or Wayland) does not by
> default mean you want emulated input to happen through those protocols.
> 

To expand on this, a side channel can be technically relatively multi
purpose, e.g. PipeWire, which is used both for screen casting access,
camera access, and eventually microphone input, sound output etc.

To make this possible, the service provider (PipeWire) needs to
integrate with the sandbox eco system some how, and PipeWire does this
by allowing two phased initialization.

This could in theory be applied to Ei sessions too.

To summarize how access management is handled when using PipeWire, there
are currently two scenarios: screen cast and camera.

For screen casting, the portal sets up screen cast PipeWire streams.
It then creates a PipeWire remote with *only* access to these streams and
nothing else. As permissions can by the client only be restricted, and
never eased, a application cannot undo the restrictions and will thus
only ever see the screen casts it opened using the opened PipeWire file
descriptor.

For camera access, the portal opens up a PipeWire remote, it sets some
metadata ("app-id", and allowed "media roles" (i.e. "Camera")), then
locks down permission to not grant access to anything at all.

It then relies on the PipeWire session manager to know that a remote is
opened by the portal, and then open up access to devices by looking at
the metadata the portal set as well as what's in the permission store.

> 
> > > 2) It provides, using backends, methods for implementing user
> > > interactive permission management. For arbitrary file system access,
> > > this may involve e.g. opening a file using a file selection dialog, or
> > > for screen casting this may mean actively choosing what part of your
> > > screen should be shared.
> > 
> > Right, what the basic idea of the portals is, is pretty clear to me.
> > Basically it pipes requests of sandboxed apps through an
> > authentication and permission system in case they want to interact
> > with outside their sandbox. It's more about the cases where they
> > don't. A Wayland client in a sandbox interacts via Wayland protocol
> > with the outside of their sandbox, but that's not going through
> > xdg-portals, or is there a permission to block it from that?
> 
> For flatpak at least, you need to explicity allow wayland access
> (--socket=wayland) at which point it's basically a wayland-sized hole in the
> sandbox.
> 
> > > 3) It manages remembered access, using a common permission store[0]. For
> > > example if an application was permanently denied camera access, the
> > > portal will know about this and not query the portal backend.
> > 
> > Yea, I quickly did an online search for a GUI to it and found [2]
> > which looks awesome. Also answers my question above: there exist
> > permissions to block a sandboxed app from Wayland or X11. But I assume
> > a user won't be asked for every new GUI app he installs if it is
> > allowed to show some pixels and instead the permission is by default
> > set to allowed. Looking back at it the GUI app should set this as a
> > permission in their Manifest file, which is like a default on install.
> 
> It does indeed ask, or at least tell you:
> 
> $ flatpak install --user org.gimp.GIMP
> Looking for matches…
> Found similar ref(s) for ‘org.gimp.GIMP’ in remote ‘flathub’ (user).
> Use this remote? [Y/n]: y
> 
> org.gimp.GIMP permissions:
>     ipc      network      x11      file access [1]      dbus access [2]      tags [3]
> 
>     [1] /tmp, host, xdg-config/GIMP, xdg-config/gtk-3.0, xdg-run/gvfs
>     [2] org.freedesktop.FileManager1, org.gtk.vfs, org.gtk.vfs.*
>     [3] stable
> 
> ...
> 
> Proceed with these changes to the user installation? [Y/n]: 
> 
> 
> So the built-in permissions for gimp include x11 and direct access to the
> org.freedesktop.FileManager1 DBus API. To re-iterate my above point -
> because gimp has x11 access, the sandbox cannot stop it from emulating input
> via XTEST. Likewise, if wayland was parts of the permission you cannot
> arbitrate access to wayland protocols through a portal.
> 
> For the rest I'm going let Jonas answer since there are some questions in
> there where I'm not sure my answer would be correct :)
> 
> Cheers,
>    Peter
> 
> 
> > > In the diagram you can see 5 "interactions" between different components
> > > that takes place, (1), (2), (3), (4) and (5), resulting in the side
> > > channel (C).
> > >
> > > (1) is the application making a request to be able to inject input
> > > events.
> > >
> > > (2) is the portal, having authenticated the application the request came
> > > from and verified the metadata, does a method call on behalf of the
> > > application to the portal backend implementation to check with the user
> > > then maybe open an Ei session.
> > >
> > > (3) is the portal backend, possibly having queried the user about
> > > permissions, opening a new Ei session.
> > >
> > > (4) is the backend returning from the method call done in (2), possibly
> > > with an open file descriptor to a Eis session.
> > 
> > The portal backend creates the open file descriptor, something the
> > sandboxed app can't do per se. In general a sandboxed app can not
> > create arbitrary sockets, correct?

The portal backend would create a session of whatever service that
provides a by-default blocked functionality. The fact that it involves
opening file descriptors is just because passing such objects around is
a convenient way to open up channels that can be passed across the
sandbox barrier.

A sandbox can do pretty much anything inside the sandbox, including
creating or connecting to arbitrary sockets, but it has limited view and
effect on anything outside of the sandbox. It would for example not be
able to see /run/user/1000/ei-0 to open it directly, it needs the portal
to open that socket, and pass the open file descriptor, after having
authenticated and authorized the request.

Or in D-Bus terms, a sandbox wouldn't, unless explicitly allowed, be
able to see e.g. a org.freedesktop.Ei D-Bus API on e.g. the
org.gnome.Mutter name on the bus, which e.g. a GNOME portal backend
might use to open the Ei socket.

> >
> > > (5) the portal responds to the requests made from the application in
> > > step (1). This response is then used to establish (C). See below.
> > >
> > > (C) is the newly established Eis channel where the application can
> > > inject input using libei, and the compositor being able to process
> > > injected input in a similar way to how it processes libinput events.
> > >
> > > (p) and (p') corresponds to permission store interaction (where p' is
> > > optional but recommended). Either up front by the portal, or e.g. a
> > > compositor may be permission store aware, so that it can terminate a
> > > session that changes permission after access was originally granted.
> > > This is for example how PipeWire handles camera access being revoked.
> > 
> > What do you mean by "terminate a session that changes permission"? Who
> > changed the permission in the first place? The user? For example in
> > the Flatseal UI by switching one app off again?

Yes, for example. Anything that pokes at the permission store, be it a
GUI like Flatseal, or GNOME Settings, some command line tool, that in a away
revokes a previously granted access could be respected immediately by
whatever service is providing a certain functionality, by listening to
permission store changes.

> > 
> > > Today, unsandboxed applications are treated on a per case basis. In some
> > > cases (e.g. screen casts, screenshots), the only effect is that the name
> > > of the application is not presented as part of a dialog, while in other
> > > cases it simply defaults to some policy e.g. deny or grant.
> > 
> > Do you mean non-sandboxed applications that still try to communicate
> > with xdg-desktop-portal? They could also just talk directly to your
> > Mutter dbus interface or does it only allow connections from
> > xdg-desktop-portal's dbus "address"? If yes, this dbus
> > address/name/identifier is not fakable?

A unsandboxed application can talk to any API, be it D-Bus, Wayland, X11
or anything else, as it sits in the same "security" layer as the
platform itself. Sometimes an API provider can add "restrictions" that
are trivial to circumvent, as an application outside of a sandbox can do
pretty much anything a user level system component can.

GNOME Shell currently only hinders non-portal communication from calling
methods on org.gnome.Shell.Introspect using name resolution, but it's by
no mean a real protection from anything. The only way to actually do
that is not exposing org.gnome.Shell.Introspect to a sandboxed
application.

> > 
> > > The permission store itself doesn't require entries to have a sandboxed
> > > application ID, but the portal itself currently doesn't have a way to
> > > identify an application that isn't running inside a sandbox. Thus, if a
> > > Ei session policy default is to ask, for a portal to be able to remember
> > > the permission for an unsandboxed application, we would have to add a
> > > way for the portal to blindly trust some ID or key of some sort the
> > > application provides.
> > 
> > What about the portal in this case being able to get additional
> > information from other "trusted actors" like the Wayland server? In
> > case of Xwayland the compositor could hand over a secret key at the
> > start that Xwayland then can provide for queries to xdg-desktop-portal
> > and then check back for every request with the compositor if the app
> > that states to be "Xwayland" provided indeed this key.

Not sure I understand this part. The idea would be that Xwayland opens
Ei sockets via the portal; it doesn't need to involve the compositor
here. How exactly it tries to reuse Ei sessions and when to open new
ones is an open question, but I don't think it needs to involve secrets
from the compositor. Xwayland, the compositor and xdg-desktop-portal
likely sits in the same "layer", so they have access to the same things,
and can be made to trust each other to some degree.

The reason for going via the portal is partly for it to have a way to
integrate with the world of sandboxes, in an environment where that is
the only portable way to get hold of an Ei session, but in theory, if
you don't care about sandboxing, Xwayland could provide unlimited access
would the environment it runs in provide a way for that.

> > 
> > >
> > > Xwayland would be considered an unsandboxed application, even if the
> > > application a XTEST request came from was sandboxed. Exactly how to deal
> > > with this, is an open question, e.g. whether to treat all "X11"
> > > applications as one giant blob, or whether to distinguish between them,
> > > depends on in what ways we would be able to let the portal trust
> > > metadata coming from unsandboxed applications.
> > >
> > 
> > From my understanding there are basically three important client classes:
> > * Unsandboxed app
> > * Sandboxed Wayland app
> > * Sandboxed X11 app
> > 
> > Some random unsandboxed app can do whatever it wants but some system
> > interfaces might be only accessible by xdg-portals like certain dbus
> > addresses for example screencasting in Mutter (if that's not true tell
> > me) and reading Peter's first email also libei is supposed to behave
> > this way in a Gnome session. The question is in this case if such apps
> > should be able to gain access to the system interfaces that are
> > accessible only by xdg-portals through its permission system.

An unsandboxed application "shouldn't" get access to privileged services in
a non-portal way, but it can, because it's not sandboxed, so it can do
whatever it wants. Sometimes application developers use supposedly
private system API, but that's rather unfortunate. For example there are
applications out there that use org.gnome.Screenshot instead of
org.freedesktop.portal.Screenshot, and that's bad, but also hard to
prevent without also making it less useful as a system component.

Due to this, there are applications that has suddenly got degraded
funtionality in some way when upgrading the OS and some system API that
noone externally was supposed to be using changed or disappeared. A
recent example of this is org.gnome.Shell.CalendarServer, an API exposed
by a tiny private service process that helps GNOME Shell talk to
evolution data server.

> > 
> > Sandboxed Wayland apps are properly secure as long as there are no
> > holes created by not secure Wayland protocol extensions.

Right.

> > 
> > Sandboxed X11 apps are as secure as unsandboxed Xwayland is contained
> > from the outside system since the X11 protocol design is per se not
> > secure. So if Xwayland is not contained properly and has access to
> > certain critical/global interfaces this could then be used as an
> > attack vector from inside the sandbox to the outside. As an example a
> > rogue sandboxed Xwayland app could fake input if Xwayland was allowed
> > in general access to libei (that also holds for unsandboxed X11 apps,
> > but for sandboxed ones it's more grave). On the other side if the
> > access is asked for by Xwayland on a per-Xwayland-client-basis with
> > Xwayland checking window name, PID (?) or something similar a rogue
> > client could still try to gain access by posing as a different client
> > that was already allowed for example by changing its window title and
> > maybe PID (would this at all be doable for a sandboxed app?).
> > 
> > With Xwayland having access to libei in some way I think at least for
> > sandboxed X11 apps there must be no possibility for tampering.
> > Unsandboxed X11 clients are not so important in comparison. If a
> > non-sandboxed X11 client gains access to input emulation through
> > Xwayland then so be it. Do you agree with this sentiment?

To be honest, I don't think we should consider granular access to X11
clients, sandboxed or not, to be something that adds granular
"security", but rather looking at it from a user experience point of
view.

For example, we should be able to trust Xwayland not to go rogue and
open Ei sockets when, by the permission store, portal, etc, told not to.
In this way, we can trust that an application won't be able to inject
input if we tell Xwayland that it may not open Ei sessions.

But I'm not sure we should start relying on Xwayland to authenticate and
authorize on a per client basis. We could add granularity to avoid
getting annoyed by some "clever" application taking over mouse, while
still allowing some legacy remote control application to work, but I
think it's a bit of a stretch to consider this related to "security".

To make it possible to "secure" sandboxed X11 using applications, I
think we would need to run Xwayland inside the sandbox. This would
require compositors to handle being window managers to N X servers etc,
which I suspect is a non-trivial task for most. But by then, the portal
would for free know what application a request came from, as Xwayland
would technically be part of sandbox.


Jonas


[0] https://docs.flatpak.org/en/latest/portal-api-reference.html#gdbus-org.freedesktop.portal.Documents



> > 
> > [1] https://docs.flatpak.org/en/latest/sandbox-permissions.html#sandbox-permissions
> > [2] https://www.omgubuntu.co.uk/2020/02/flatseal-manage-flatpak-permissions
> > 


> > >
> > > Jonas
> > >
> > >
> > > [0] https://github.com/flatpak/xdg-desktop-portal/wiki/The-Permission-Store
> > > >
> > > > [1] https://gitlab.com/romangg/kwinft/-/commits/libei
> > > > [2] https://gitlab.freedesktop.org/xorg/xserver/-/merge_requests/431
> > > > [3] https://gitlab.freedesktop.org/xorg/xserver/-/merge_requests/465
> > > >
> > > > > The aim is that a client can simply iterate through all of the options until
> > > > > finds a connection. Once that's found, the actual code for emulating input is
> > > > > always the same so it's trivial to implement a client that works on any
> > > > > compositor that supports some backend of libeis.
> > > > > The server part only needs to care about the negotiation mechanisms it
> > > > > allows, i.e. GNOME will only have dbus/portal, sway will only have... dunno,
> > > > > fd exchange maybe?
> > > > >
> > > > > Next: because we have a separate channel for emulated input we can hook up
> > > > > XTEST to use libei to talk to a compositor. I have a PoC implementation for
> > > > > weston and Xwayland:
> > > > >   https://gitlab.freedesktop.org/whot/weston/-/commits/wip/eis
> > > > >   https://gitlab.freedesktop.org/whot/xserver/-/commits/wip/xwayland-eis
> > > > > With that xdotool can move the pointer. Note this is truly the most minimal
> > > > > code just to illustrate the point but you can fill in the blanks and do
> > > > > things like the compositor preventing XTEST or not, etc.
> > > > >
> > > > > This is all in very early stages with very little error checking so things
> > > > > will probably crash or disconnect unexpectedly. I've tried to document the
> > > > > API to make the intentions clear but there are still some very handwavy
> > > > > bits.
> > > > >
> > > > > Do let me know if you have any questions or suggestions please though.
> > > > >
> > > > > Cheers,
> > > > >   Peter
> > > > > _______________________________________________
> > > > > wayland-devel mailing list
> > > > > wayland-devel at lists.freedesktop.org
> > > > > https://lists.freedesktop.org/mailman/listinfo/wayland-devel
> > > > _______________________________________________
> > > > wayland-devel mailing list
> > > > wayland-devel at lists.freedesktop.org
> > > > https://lists.freedesktop.org/mailman/listinfo/wayland-devel