[GSoC] Some questions about wayland clients with no outputs present

Mon May 2 10:49:04 UTC 2016

On 30.04.2016 22:14, Bryce Harrington wrote:
> On Sat, Apr 30, 2016 at 05:20:09PM +0200, Armin Krezović wrote:
>> Hi all,
>>
>> Now that I've analyzed the weston codebase a bit, I'm ready to
>> bombard you people with some questions :).
>>
>> For those that aren't yet aware, I'll be working on making weston
>> able to run with no outputs and survive with all outputs plugged
>> out at runtime.
>>
>> For starters, I've simply commented out the output backend load code
>> in src/main.c to see what would happen. To my surprise, the compositor
>> didn't crash, but the desktop-shell did.
>>
>> I'm currently guessing that it crashes because the clients that use
>> clients/window.c code look for outputs in the registry. Of course,
>> there are none because the backend that updates the global registry
>> isn't loaded.
>>
>> Now, my conclusion is: In order to make weston work without any outputs
>> present is to make clients work without any outputs present. This will
>> probably require people to update their toolkits and apps.
>>
>> As I've originally proposed, I see two different ways in achieving this:
>>
>> First and easiest way is use of a virtual output. I could modify the
>> DRM backend to add a virtual output if no physical outputs are found
>> and remove the output when a physical output is connected. Add similar
>> behavior on phyisical screen unplug. If there are none left, add a virtual
>> output. If I remember correctly, Pekka once noted that DRM backend already
>> supports output hotplugging, more or less, but it doesn't support the
>> primary output unplugging.
> 
> It happens that the x11 backend is able to run with 0 outputs.  Perhaps
> it would be simpler rather than defaulting to virtual outputs to just
> have the drm backend also run with 0.  I suppose in that case it would
> also skip the rendering paths.

Thanks for the info, I'll take a look at the X11 backend.

> 
> But virtual outputs is not a bad idea, it could be useful for testing or
> perhaps to extend one's rendering beyond ordinary hardware capabilities
> or something.  But it should be user-controlled via commandline options
> or config file options.
> 

I was thinking about using the virtual outputs when no physical outputs
are present, and keep them out of the way otherwise. Sure, a command
line switch/config option is useful, especially for the test suite (Pekka
proposed something similar).

> Keep in mind also potential security issues if we render clients to
> outputs the user isn't aware of or able to actually see.
> 

See above.

>> Now, the first question is: Even if this way is picked, would clients or
>> something else require much work to make them behave correctly when
>> the primary output is unplugged?
> 
> I don't think so.  When the screen blanking kicks in, the weston
> compositor actually removes all the client surfaces.  I would guess
> something analogous could be done here.  The client surfaces simply
> become a state where rendering isn't happening.
> 
> I wrote up some notes on details of how the screen blanking functions in
> detail in my quest to understand how that works.  I can send it to you
> if you'd like; I'm wondering if perhaps the 0-output situation could be
> considered analogous - the clients are still connected but nothing is
> actively rendering anywhere.  I suppose in a way, at least from a client
> perspective, powering a monitor off vs. actually unplugging isn't all
> that totally different.
> 

Yes, please send me the notes you have. I welcome anything that may
help me achieve my goal.

> I also posted a set of patches to allow clients to request their monitor
> not be powered off (e.g. if the client is displaying a presentation or
> movie).  These patches provide for allowing outputs to individually come
> and go.  They're implementing this at a higher level than you'll be
> working, but might give a useful perspective.  I can send you the link
> if you're interested.
> 

I don't think it's related, but sure, why not. As noted above, anything
is welcome that helps me implement this feature.

>> The second and more harder way is to simply cope with no outputs present and
>> watch for output plugging/unplugging. This would require clients to watch for
>> output changes and take apropriate action.
> 
> I would think this is not something the clients need to take action on.  
> E.g. for the screen blanking, when an output becomes inactive the client
> doesn't need to watch for that, the compositor simply stops directing
> them to render.
> 

Hm, thanks for the info. I kinda got confused between desktop-shell/shell.c
and clients/desktop-shell.c here, which is what lead me to this conclusion.

Now, to confirm, the shell is the X window manager equivalent of the wayland
world? Is it responsible for placing windows and moving them around? If so,
my work has suddenly become way more easier.

> The slight but crucial difference, however, is that when a screensaver
> kicks on, you don't want the clients to rearrange or move around.  You
> want everything to come back exactly as it was; whereas with output
> unplugging, you probably *do* want things rearranged.  Unfortunately the
> tricky thing there is that how to rearrange things is going to be a
> policy thing.  I should imagine different *shell* implementations could
> desire setting different types of settings.  Neither the backends nor
> the clients should be making that policy, just the shells.
> 
> The fullscreen shell is simplest - there's just one client so it is just
> resized to the new number of outputs, or if 0 outputs just stops
> rendering, and restores when some output gets attached.  The desktop
> shell perhaps should force clients to be moved off of the removed
> output(s), maybe just to some randomly chosen spot.  Extra snazzy would
> be if it could rearrange the clients to look nice on the remaining
> outputs (but that's almost certainly beyond scope for your project).
> Other shells like IVI would probably have their own desired behavioral
> policies...
> 

This is why I asked what should be done on output unplugging. I see that
window placement isn't that important, but I'd still like to try to
arrange the windows symetrical to the unplugged screen. I am unsure if
they should be scaled/resized in case of, ie, moving from a screen with
bigger resolution to a screen with smaller one. What do other
compositors/shells do?

> I would imagine that Weston would want to exhibit the same general
> behavior when outputs are changed, for any backend.  So if you're
> running desktop shell on the drm, x11, headless or other backends and
> remove one output, then whatever happens as a result should always
> the same result regardless of backend.
> 
> So, the backends should indicate to the compositor (and thence to the
> enabled shell) that the output change has occurred, and then the shell
> has the responsibility to rearrange the client surfaces as it wishes.
> Perhaps you'd like to focus your scope on the backend side of things,
> make sure the communication path *to* the shell is all working, but
> leave the layout logic as outside scope to be done as followup work.
> 

I think that desktop-shell already supports rendering on multiple outputs
(as confirmed by plugging in a second monitor) and stopping any rendering
from a certain output when it gets removed. It still needs to properly handle
screen blanking and screensaver when all outputs are gone. The things will
change when I start working on second part of my GSoC, which involves
extended screens.

>> The second question then might be: What should clients do in such case?
>>
>> 1. Do they only need to schedule redraw and toggle back fullscreen if it was on?
> 
> Hope my ramblings above stumbled into an answer for this.
> 
>> 2. Do they need to resize/scale relative to the size of new screen? (eg, a maximized
>> app on 1920x1080 screen would not be entirely visible on 1600x900 one).
>>
>> 3. Is there something as "move an app to different screen" that can be issued
>> by compositor? Or do apps need to do that too?
> 
> #2 and #3 are Weston shell policy, so there won't be a single answer;
> potentially each shell could do it differently.  Again, I suggest
> focusing just on making the underpinnings work right, and maybe pick one
> shell (likely the desktop shell but pq might prefer you focus on IVI or
> something) as a demo/reference implementation.
> 

As noted above, I'd like for clients to be on exactly similar spot on the
new screen, even after all outputs are gone. This might require preserving
client layout somewhere, so would it be okay?

As for the shell choice, I haven't worked on IVI shell yet, or even used it.
I'm more familar with the desktop shell, but I may need to ask more questions
about some terms used in the implementation that I don't yet understand.

>> 4. Is there anything special needed for XWayland or is it just another wayland client?
> 
> I would suggest leaving Xwayland outside scope for your project, I
> think you're going to have plenty on your plate as it is.
> 

Fair enough.

> Bryce
> 

Thanks for taking your time to respond. It means a lot.

Armin.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20160502/a6326f4a/attachment.sig>