<div dir="ltr">On Mon, Aug 26, 2013 at 3:48 AM, Cedric BAIL <span dir="ltr"><<a href="mailto:moa.bluebugs@gmail.com" target="_blank">moa.bluebugs@gmail.com</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">Hello,</p>
<p dir="ltr">Cedric Bail</p><div><div class="h5"><br>
On Aug 26, 2013 2:11 AM, "David Herrmann" <<a href="mailto:dh.herrmann@gmail.com" target="_blank">dh.herrmann@gmail.com</a>> wrote:<br>
><br>
> Hi<br>
><br>
> On Fri, Aug 23, 2013 at 11:55 PM, Jason Ekstrand <<a href="mailto:jason@jlekstrand.net" target="_blank">jason@jlekstrand.net</a>> wrote:<br>
> > Hello All,<br>
> > I am in the process of picking back up the old idea of system compositors.<br>
> > I am not, at the moment, looking for a review of the code; simply a review<br>
> > of the concept and the proposed protocol. If you would like to look at my<br>
> > implementation or try it out, it can be found in the system-compositor<br>
> > branch of my personal [weston fork on github][1].<br>
> ><br>
> > What follows is what I envision for system compositors. (Others may have a<br>
> > different idea which is why I'm writing this RFC email.) The basic idea<br>
> > behind a system compositor is to provide an interface for compositors whose<br>
> > only job it is to display other compositors or other stand-alone<br>
> > full-screen interfaces. I image three primary purposes for system<br>
> > compositors:<br>
> ><br>
> > 1. As an abstraction layer. Every time someone wants to write a RDP<br>
> > server, VNC server, android backend, or the like the standard answer is<br>
> > "write it as a weston plugin". The problem with this is that, to my<br>
> > knowledge, none of the major desktop environments (GNOME, KDE, EFL,<br>
> > etc.) plan to run their compositor as a weston shell plugin. This makes<br>
> > a weston plugin a poor solution in the long term. On the other hand, a<br>
> > system compositor is fairly simple to implement and provides a backend<br>
> > to any compositor that can run on top of a system compositor.<br>
><br>
> KDE planned on using weston as underlying compositor. But I doubt<br>
> GNOME or EFL want to use something like that..</div></div><p></p>
<p dir="ltr">Indeed, Enlightenment is not planning to use Weston. But if a common library dealing with all those drm / kms, android, rpi,... did exist, I am sure we would use it and contribute to it. I am not a big fan of adding a system compositor as it will impact speed and power consumption. As long as we can avoid it, and work on systemd / logind seems to go this way, I think it would be better.</p>
</blockquote><div style>My fear here is that if we try to write it as a library, we will end up writing an entire toolkit-style drawing abstraction. Whatever abstraction we come up with will not be the same as those used in Qt/EFL/Gtk, and so it won't be used for the exact same reason they're not using Weston. Maybe that's not the case, but I'm skeptical. Another option would be to write some sort of common plug-in API that all of the compositors support but I think it will suffer from some of the same issues.<br>
</div><div style><br></div><div style>Please not that I do NOT expect mutter/kwin/E to use this as a back-end for DRM/KMS. There are two reasons for this. First is a performance reason that you have already mentioned. For the standard use-case (desktop environment on a local machine) we want to streamline things as much as possible so a library or doing it yourself is a better option. The second reason is that, in order to make things sufficiently simple that we can take some of the load off of smaller compositors and/or simple fullscreen apps such as terminal emulators, we can't support everything that KMS can do. Major compositors will quite likely want more fine-grained control and that won't be possible if we keep things simple. Maybe there is a perfect balance somewhere, but I don't know what it is.</div>
<div style><br></div><div style>As far as my main use-cases go, I don't think the performance argument really works. In the Android app case (what I'd like to do eventually), it has to be out-of-band anyway in order to cross between systems. For something like RDP/VNC, there is enough overhead anyway that I question whether or not the context-switch will make any noticeable difference.</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
<p dir="ltr">> > 2. Simple full-screen clients. With standard VTs going the way of the dodo<br>
> > bird (see also [David Herrmann's work on logind][2]), we will need a way<br>
> > for displaying simple full-screen programs such as splashscreens without<br>
> > every single one of them knowing how to talk DRM/KMS directly. David<br>
> > has proposed to use a system compositor (which he calls wlsystemc in his<br>
> > post) so these types of programs can be written as simple wayland<br>
> > clients. My github repository contains the (almost trivial)<br>
> > modifications to simple-shm that make it in to one such client.<br>
><br>
> Yay, I appreciate someone picking up this idea \o/<br>
><br>
> > 3. As a DRM/KMS backend for other compositors. Manually dealing with<br>
> > DRM/KMS isn't actually all that many lines and all of the big<br>
> > compositors (mutter, Kwin, EFL, etc.) will do it themselves. However,<br>
> > for people who want to write their own simple compositor, it can be a<br>
> > bit tricky to get right and raises the barrier to entry. The protocol<br>
> > I'm proposing is sufficiently powerful to provide most of the basic<br>
> > multi-output support and modesetting that is needed for a simple<br>
> > compositor. This means that all a potential compositor writer has to do<br>
> > is write a system compositor client and they can let someone else get<br>
> > the KMS details right.<br>
><br>
> Manually dealing with DRM/KMS _is_ hard. Yes, the basic setup is easy,<br>
> but I hate the fact that we now end up with everyone doing it on their<br>
> own. Advanced things can get pretty complex, like GPU selection,<br>
> multi-GPU handling, GPU hotplugging, CRTC hotplugging, planes,<br>
> cursors, DRM-Master, DPMS, DRM properties, atomic modesetting,<br>
> render-nodes, .. ugh, this list can get soo long. I am a little bit<br>
> scared that EFL or Gnome get it wrong, but ok, not my deal.<br>
><br>
> > While it sounds like a big task, implementing a system compositor isn't<br>
> > that bad. The simplest system compositor is one that can simply display<br>
> > surfaces. In order to do that, you need to implement the following<br>
> > interfaces:<br>
> ><br>
> > * wl_compositor<br>
> > * wl_region<br>
> > * wl_shm (along with wl_shm_pool and a SHM-based wl_buffer)<br>
> > * wl_surface<br>
> > * wl_output<br>
> ><br>
> > None of those are particularly difficult or complicated to implement. The<br>
> > hardest is probably wl_surface and that's not terrible.<br>
> ><br>
> > For input you have two options. The first, most obvious option is to<br>
> > implement wl_seat and its child interfaces. Second, I have considered the<br>
> > idea of a wl_raw_input interface (only for system compositors) that simply<br>
> > provides evdev file descriptors to the client. This would allow for raw<br>
> > input processing without having to worry about things like weston-launch.<br>
> > Also the system compositor *may* be able to handle the FD muting etc. for<br>
> > playing nice with logind. (I'm not sure on that one yet, I'd have to ask<br>
> > David.)<br>
> ><br>
> > One more note on input: I do NOT expect the system compositor to do any<br>
> > significant input processing. Even if the child compositor is getting its<br>
> > events through wl_seat, I expect them to be about as raw as possible. In<br>
> > particular, this is not where things like pointer acceleration should be<br>
> > handled. Once we get a pointer_grab interface working, I expect any<br>
> > compositor that runs on top of a system compositor to attempt a pointer<br>
> > grab almost immediately and do its own pointer handling from there.<br>
> > Otherwise it is impossible for the client compositor to re-arrange the<br>
> > outputs without additional protocol.<br>
> ><br>
> > Along with the basics as described above, a system compositor could<br>
> > optionally implement additional interfaces to provide aditional<br>
> > functionality. For example:<br>
> ><br>
> > * A RDP system compositor could implement wl_data_device to allow for<br>
> > drag-and-drop and clipboard sharing between host and client.<br>
> ><br>
> > * A system compositor could expose wl_subcompositor and use subsurfaces to<br>
> > allow the client compositor to take advangage of hardware compositing.<br>
> > Note that this is not as simple as it sounds because hardware frequently<br>
> > has limits on the number, size, and placement of such overlays and there<br>
> > is currently no way to communicate that information through the<br>
> > subsurface protocol. Therefore, the system compositor could find itself<br>
> > doing a full composite anyway.<br>
> ><br>
> > * A wl_user_switcher interface could be created for login managers. This<br>
> > steps on logind a bit, but someone may find it useful.<br>
> ><br>
> > That is about all I have to say for now on the subject. If you want to<br>
> > check out my current implementation you can look at my github as I said<br>
> > above. As of right now, I have multi-monitor weston running inside of my<br>
> > weston-based system compositor. Also, I have simple-shm modified to act as<br>
> > a simple system compositor client if finds the wl_system_compositor global.<br>
> > The only mode that works right now is "center" but I'm working on getting<br>
> > the others to work.<br>
> ><br>
> > I appreciate any questions or comments you may have on either the main<br>
> > ideas (above) or the protocol (below).<br>
><br>
> I really like having an intermediate layer for simple tasks. Some<br>
> layer that is well maintained and allows things like plymouth to run<br>
> on top of it. However, I dislike calling it system-compositor. Let me<br>
> explain why:<br>
><br>
> In a session there are several different running daemons. Each of them<br>
> provides a different service to ease device usage. This includes<br>
> pulseaudio, dbus, polkit, colord, .. and of course the<br>
> session-compositor (xserver, weston, ...). But all of these run<br>
> _inside_ the session. If properly done, they don't need any super-user<br>
> privileges. If we switch sessions (whether via logind or via VT<br>
> doesn't matter), we can simply change device-access restrictions and<br>
> the new session can start running. We handover device access from one<br>
> session to another. My work on logind doesn't change this or reinvent<br>
> it, it only tries to make it _safe_ and _reliable_.<br>
><br>
> The idea behind a system-compositor was to provide a system daemon<br>
> that runs _outside_ a session. Its sole responsibility is to control<br>
> access to graphics and input hardware. So session-compositors no<br>
> longer access hardware directly, but instead tunnel it through the<br>
> system-compositor. But this means, the system-compositor must know of<br>
> session-switches and correctly display only the session-compositor of<br>
> the active session. However, session-switching is controlled by<br>
> logind, so the system-compositor gets the session-switch notification<br>
> _after_ the session was actually switched, making this kind of racy<br>
> (but still ok!).<br>
> The bigger problem is, the system-compositor is not part of a session<br>
> so it has to be active *all the time*. You cannot have some sessions<br>
> using the system-compositor and other sessions doing it the old way.<br>
> You cannot do device-access handover from the system-compositor to a<br>
> self-hosting or legacy session. This would require ugly racy hacks and<br>
> conflict with VT!<br>
> But if the system-compositor is always active, you cannot use VTs in<br>
> text-mode. Because VTs in text-mode access graphics hardware<br>
> *directly*.<br>
><br>
> With this in mind, we ditched the idea of a system-compositor. But<br>
> that doesn't mean the idea of tunneling graphics access is wrong! In<br>
> fact, I like it a lot, but please don't call it system-compositor! A<br>
> system-compositor is what Ubuntu people are trying and RAOF just<br>
> recently made me aware that they ran in exactly the problems I just<br>
> mentioned. So please don't do that.<br></p></div></div></blockquote><div style>I Agree. I'm not really trying to create a system compositor. More of a "put stuff on a screen somewhere" compositor. That said, it would be nice if it could also handle whatever is needed to talk to logind. I'm not sure how much there is though.</div>
<div style><br></div><div style>As far as names go, I'm open to suggestions. Here are a few that come to mind:</div><div style><br></div><div style> * wl_fullscreen_shell</div><div style> * wl_metacompositor</div><div style>
* wl_canvas</div><div style> * wl_terminal</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5"><p dir="ltr">
> So back to your proposal. I'd like to see something like you did but<br>
> as a session-compositor (call it whatever you want ;)). So a session<br>
> that doesn't want to deal with DRM directly (like for instance<br>
> gdm/xdm/kdm) could avoid starting an xserver or weston and start your<br>
> session-compositor instead. It then displays it's content via the<br>
> standard wayland client APIs on this compositor. But this compositor<br>
> imho should run in a session. So it does *not* allow clients from<br>
> different sessions to connect. It is *no* system compositor. It's just<br>
> a daemon which provides graphics-access to the session. It may even<br>
> allow switching between multiple surfaces. And if you continue this<br>
> thought, you will notice that it is nothing more than weston but with<br>
> a *very* reduced wl_shell. Precisely a wl_shell that displays only one<br>
> surface at a time.</p></div></div></blockquote><div style>Agreed. Personally, I think multi-client is a bit over-the-top for now but maybe in the future. I hope that it's going to be light enough that running it as a daemon is unnecessary but we can one one instance per client.<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
</div></div><p dir="ltr">What would be the benefit over a library that does the job in the compositor/single application ?</p></blockquote><div style>See above </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<p dir="ltr"></p><div><div class="h5">> So if the protocol is well-defined, we could standardize some path<br>
> (eg., /run/user/<num>/wayland-fullscreen-<sid>.socket) which is<br>
> provided by such a session-compositor. gdm could then during startup<br>
> check whether this path exists (that is, such a compositor is running<br>
> in its session) and use it to access graphics-devices. If the API<br>
> allows multiple surfaces, a separate "system-log"-process could run in<br>
> the _same_ session and use this compositor to display the current<br>
> system-log. Your compositor could react to keypresses like alt+1 or<br>
> alt+2 (or ESC) to switch between the surfaces. That is, the gdm<br>
> session could provide the system-log just one key-press away. Both in<br>
> the same session! You don't have to run a full-blown<br>
> VT/kmscon/whatever-log for that on another session on VT12..<br></div></div></blockquote><div><br></div><div style>Yeah, not having to burn VT's for things like logs would be nice. I'm not sure what I think of leaving a standardized socket open. For a lot of cases, I think I prefer either the fullscreen compositor launch the other application (so "wlvnc --options -- gnome") or the primary compositor (i.e. mutter) launch the fullscreen compositor, hand it the connection as a env var and start drawing on it. This would allow a fullscreen compositor to run as a sort of plugin to the bigger compositor if it wants to handle EGL itself but mirror to VNC or something like that.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
><br>
> > Protocol follows:<br>
> > -----------------<br>
> ><br>
> > <protocol name="system_compositor"><br>
> > <interface name="wl_system_compositor" version="1"><br>
> > <description summary="Displays a single surface per output"><br>
> > Displays a single surface per output.<br>
> ><br>
> > This interface can only be bound to by one client at a time.<br>
> > </description><br>
> ><br>
> > <enum name="fullscreen_method"><br>
> > <description summary="different method to set the surface fullscreen"><br>
> > Hints to indicate to the compositor how to deal with a conflict<br>
> > between the dimensions of the surface and the dimensions of the<br>
> > output. The compositor is free to ignore this parameter.<br>
> > </description><br>
> > <entry name="default" value="0" summary="no preference, apply default policy"/><br>
> > <entry name="scale" value="1" summary="scale, preserve the surface's aspect ratio and center on output"/><br>
> > <entry name="driver" value="2" summary="switch output mode to the smallest mode that can fit the surface, add black borders to compensate size mismatch"/><br>
> > <entry name="fill" value="3" summary="no upscaling, center on output and add black borders to compensate size mismatch"/><br>
> > </enum><br>
> ><br>
> > <request name="present_surface"><br>
> > <description summary="present surface for display"><br>
> > This requests the system compositor to display surface on output.<br>
> > Each client of the system compositor can have at most one surface<br>
> > per output at any one time. Subsequent requests with the same<br>
> > output replace the surface bound to that output. The same surface<br>
> > may be presented on multiple outputs.<br>
> ><br>
> > If the output is null, the compositor will present the surface on<br>
> > whatever display (or displays) it thinks best. In particular, this<br>
> > may replace any or all surfaces currently presented so it should<br>
> > not be used in combination with placing surfaces on specific<br>
> > outputs.<br>
> ><br>
> > The method specifies how the surface is to be persented. These<br>
> > methods are identical to those in wl_shell_surface.set_fullscreen.<br></div></div></blockquote><div><br></div><div style>It should be noted here that, as with wl_shell_surface.set_fullscreen, the fullscreen compositor is more than welcome to ignore this argument and do something else it feels to be sensible. For instance, if displaying on something with a constant screen size (in pixels) the "driver" mode may mean scale as far as you can by an integer multiple and add black borders.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
> > </description><br>
> > <arg name="surface" type="object" interface="wl_surface"/><br>
> > <arg name="method" type="uint"/><br>
> > <arg name="framerate" type="uint"/><br>
> > <arg name="output" type="object" interface="wl_output" allow-null="true"/><br>
> > </request><br>
> > </interface><br>
> > </protocol><br>
><br>
> Yepp, that's all we need. Just rename it from "system_compositor" to<br>
> "wl_fullscreen_shell" (I bet you can come up with some fancier name).<br>
> No other criticism on this proposal from my side.<br></div></div></blockquote><div><br></div><div style>Thanks for looking over it!</div><div><br></div><div style>As one other note. I'm not sure if I will really be able to write/maintain a fullscreen compositor solution. My knowledge of KMS is lacking (I'm a quick study but it still takes studying) and I don't really have the time to commit to maintaining it. That said, I am in the process of building a weston shell that provides this functionality and that should do at least for now. This should allow other compositors to take advantage of weston backends without using weston's scenegraph.</div>
<div style><br></div><div style>Thanks for all the comments,</div><div style>--Jason Ekstrand </div></div><br></div></div>