[GSoC] Some questions about wayland clients with no outputs present

[Pekka Paalanen]

On Mon, 2 May 2016 12:49:04 +0200
Armin Krezović <krezovic.armin at gmail.com> wrote:

> On 30.04.2016 22:14, Bryce Harrington wrote:
> > On Sat, Apr 30, 2016 at 05:20:09PM +0200, Armin Krezović wrote:  
> >> Hi all,
> >>
> >> Now that I've analyzed the weston codebase a bit, I'm ready to
> >> bombard you people with some questions :).
> >>
> >> For those that aren't yet aware, I'll be working on making weston
> >> able to run with no outputs and survive with all outputs plugged
> >> out at runtime.
> >>
> >> For starters, I've simply commented out the output backend load code
> >> in src/main.c to see what would happen. To my surprise, the compositor
> >> didn't crash, but the desktop-shell did.
> >>
> >> I'm currently guessing that it crashes because the clients that use
> >> clients/window.c code look for outputs in the registry. Of course,
> >> there are none because the backend that updates the global registry
> >> isn't loaded.
> >>
> >> Now, my conclusion is: In order to make weston work without any outputs
> >> present is to make clients work without any outputs present. This will
> >> probably require people to update their toolkits and apps.
> >>
> >> As I've originally proposed, I see two different ways in achieving this:
> >>
> >> First and easiest way is use of a virtual output. I could modify the
> >> DRM backend to add a virtual output if no physical outputs are found
> >> and remove the output when a physical output is connected. Add similar
> >> behavior on phyisical screen unplug. If there are none left, add a virtual
> >> output. If I remember correctly, Pekka once noted that DRM backend already
> >> supports output hotplugging, more or less, but it doesn't support the
> >> primary output unplugging.  
> > 
> > It happens that the x11 backend is able to run with 0 outputs.  Perhaps
> > it would be simpler rather than defaulting to virtual outputs to just
> > have the drm backend also run with 0.  I suppose in that case it would
> > also skip the rendering paths.  

Hi,

yes, I would much prefer dealing with zero outputs.

> 
> Thanks for the info, I'll take a look at the X11 backend.
> 
> > 
> > But virtual outputs is not a bad idea, it could be useful for testing or
> > perhaps to extend one's rendering beyond ordinary hardware capabilities
> > or something.  But it should be user-controlled via commandline options
> > or config file options.
> >   
> 
> I was thinking about using the virtual outputs when no physical outputs
> are present, and keep them out of the way otherwise. Sure, a command
> line switch/config option is useful, especially for the test suite (Pekka
> proposed something similar).

Let's keep virtual/fake outputs contained in the headless backend for
the time being.

> > Keep in mind also potential security issues if we render clients to
> > outputs the user isn't aware of or able to actually see.
> >   
> 
> See above.
> 
> >> Now, the first question is: Even if this way is picked, would clients or
> >> something else require much work to make them behave correctly when
> >> the primary output is unplugged?  
> > 
> > I don't think so.  When the screen blanking kicks in, the weston
> > compositor actually removes all the client surfaces.  I would guess
> > something analogous could be done here.  The client surfaces simply
> > become a state where rendering isn't happening.
> > 
> > I wrote up some notes on details of how the screen blanking functions in
> > detail in my quest to understand how that works.  I can send it to you
> > if you'd like; I'm wondering if perhaps the 0-output situation could be
> > considered analogous - the clients are still connected but nothing is
> > actively rendering anywhere.  I suppose in a way, at least from a client
> > perspective, powering a monitor off vs. actually unplugging isn't all
> > that totally different.
> >   
> 
> Yes, please send me the notes you have. I welcome anything that may
> help me achieve my goal.
> 
> > I also posted a set of patches to allow clients to request their monitor
> > not be powered off (e.g. if the client is displaying a presentation or
> > movie).  These patches provide for allowing outputs to individually come
> > and go.  They're implementing this at a higher level than you'll be
> > working, but might give a useful perspective.  I can send you the link
> > if you're interested.
> >   
> 
> I don't think it's related, but sure, why not. As noted above, anything
> is welcome that helps me implement this feature.
> 
> >> The second and more harder way is to simply cope with no outputs present and
> >> watch for output plugging/unplugging. This would require clients to watch for
> >> output changes and take apropriate action.  
> > 
> > I would think this is not something the clients need to take action on.  
> > E.g. for the screen blanking, when an output becomes inactive the client
> > doesn't need to watch for that, the compositor simply stops directing
> > them to render.
> >   
> 
> Hm, thanks for the info. I kinda got confused between desktop-shell/shell.c
> and clients/desktop-shell.c here, which is what lead me to this conclusion.
> 
> Now, to confirm, the shell is the X window manager equivalent of the wayland
> world? Is it responsible for placing windows and moving them around? If so,
> my work has suddenly become way more easier.

The shell plugin part, yes. The shell plugin is the window manager. The
shell helper client (clients/desktop-shell.c) is an implementation
detail of the shell plugin, where we move all drawing etc. operations
into another process. We keep weston mostly clean of any drawing code
this way, we avoid adding API for compositor-internal drawing and
buffer allocation, and we avoid stalling the compositor in drawing
operations.

> > The slight but crucial difference, however, is that when a screensaver
> > kicks on, you don't want the clients to rearrange or move around.  You
> > want everything to come back exactly as it was; whereas with output
> > unplugging, you probably *do* want things rearranged.  Unfortunately the
> > tricky thing there is that how to rearrange things is going to be a
> > policy thing.  I should imagine different *shell* implementations could
> > desire setting different types of settings.  Neither the backends nor
> > the clients should be making that policy, just the shells.

Exactly.

> > 
> > The fullscreen shell is simplest - there's just one client so it is just
> > resized to the new number of outputs, or if 0 outputs just stops
> > rendering, and restores when some output gets attached.  The desktop
> > shell perhaps should force clients to be moved off of the removed
> > output(s), maybe just to some randomly chosen spot.  Extra snazzy would
> > be if it could rearrange the clients to look nice on the remaining
> > outputs (but that's almost certainly beyond scope for your project).
> > Other shells like IVI would probably have their own desired behavioral
> > policies...
> >   
> 
> This is why I asked what should be done on output unplugging. I see that
> window placement isn't that important, but I'd still like to try to
> arrange the windows symetrical to the unplugged screen. I am unsure if
> they should be scaled/resized in case of, ie, moving from a screen with
> bigger resolution to a screen with smaller one. What do other
> compositors/shells do?

The first thing with Weston is to keep the code simple. shell.c might
be one of the messiest files in Weston, so there is a lot going on
anyway.

What to do is compositor policy. The important thing is to maintain
user's access to all windows, so the very least they would need moving.
That might even be implemented already. A nice bonus would be to resize
the windows down to have them completely visible.

Btw. don't do scaling. Just do a one-shot compositor-initiated resize,
and the clients will redraw in the new size. The shell protocol
specifications tell you what could be done and what might happen.

> > I would imagine that Weston would want to exhibit the same general
> > behavior when outputs are changed, for any backend.  So if you're
> > running desktop shell on the drm, x11, headless or other backends and
> > remove one output, then whatever happens as a result should always
> > the same result regardless of backend.

Yes, that is how it must be.

> > 
> > So, the backends should indicate to the compositor (and thence to the
> > enabled shell) that the output change has occurred, and then the shell
> > has the responsibility to rearrange the client surfaces as it wishes.
> > Perhaps you'd like to focus your scope on the backend side of things,
> > make sure the communication path *to* the shell is all working, but
> > leave the layout logic as outside scope to be done as followup work.
> >   
> 
> I think that desktop-shell already supports rendering on multiple outputs
> (as confirmed by plugging in a second monitor) and stopping any rendering
> from a certain output when it gets removed. It still needs to properly handle
> screen blanking and screensaver when all outputs are gone. The things will
> change when I start working on second part of my GSoC, which involves
> extended screens.

Seems like you are talking about weston-desktop-shell the client.
Indeed, fairly easy to confirm with e.g. the x11-backend by letting
Weston create more than one X11 window.

I forget if it was the Weston core or desktop shell plugin that
triggers blanking and powersave, and screensaver was removed some time
ago.

> >> The second question then might be: What should clients do in such case?
> >>
> >> 1. Do they only need to schedule redraw and toggle back fullscreen if it was on?  
> > 
> > Hope my ramblings above stumbled into an answer for this.
> >   
> >> 2. Do they need to resize/scale relative to the size of new screen? (eg, a maximized
> >> app on 1920x1080 screen would not be entirely visible on 1600x900 one).
> >>
> >> 3. Is there something as "move an app to different screen" that can be issued
> >> by compositor? Or do apps need to do that too?  
> > 
> > #2 and #3 are Weston shell policy, so there won't be a single answer;
> > potentially each shell could do it differently.  Again, I suggest
> > focusing just on making the underpinnings work right, and maybe pick one
> > shell (likely the desktop shell but pq might prefer you focus on IVI or
> > something) as a demo/reference implementation.

Let us focus primarily on the desktop shell indeed. Fixing and testing
the other shells is completely secondary, apart from checking they are
not broken in the process.

> As noted above, I'd like for clients to be on exactly similar spot on the
> new screen, even after all outputs are gone. This might require preserving
> client layout somewhere, so would it be okay?

Yeah in the case where the last output is unplugged, and then later a
new first output is plugged in. In that case it would make very good
sense to maintain window positions if possible. Only if any window
would end up completely outside of the visible region, then it should
be moved.

Preserving the exact layout should be simple: do nothing.

Resizing is up to you, but don't make it too complicated. We need to
review and maintain that code, and desktop shell code is not exactly in
the prime focus of weston's goals. :-)

> 
> As for the shell choice, I haven't worked on IVI shell yet, or even used it.
> I'm more familar with the desktop shell, but I may need to ask more questions
> about some terms used in the implementation that I don't yet understand.

If we would be putting the shells in order of importance for this
project, I would say:
1. desktop shell
2. fullscreen shell
3. ivi-shell

You are welcome to ignore all except desktop shell. However, if you
change Weston internal APIs, you are expected to migrate all users of
it.

> >> 4. Is there anything special needed for XWayland or is it just another wayland client?  
> > 
> > I would suggest leaving Xwayland outside scope for your project, I
> > think you're going to have plenty on your plate as it is.
> >   
> 
> Fair enough.

Yup.


Thanks,
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 811 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20160502/2cf62fce/attachment-0001.sig>