Sub-surface protocol

[Pekka Paalanen]

On Wed, 05 Dec 2012 22:45:14 -0800
Bill Spitzak <spitzak at gmail.com> wrote:

> Pekka Paalanen wrote:
> 
> > I am currently looking into sub-surfaces, first to sketch the protocol
> > extension, and I have some open questions. I decided to write an
> > exhaustive document, so we would all be on the same page, and also to
> > clarify my own thoughts.
> 
> Glad to see this is being worked on. I have been trying to figure this 
> out as well, so I have a number of comments and ideas:

Our design principles seem to be very different, which becomes obvious a
few paragraphs below.

> The main thing I would do is merge this with the popup/menu/transient 
> windows. Subsurfaces are simply child windows that are "stuck" to the 
> main one: the shell is not allowed to place another window between them 
> and their parent. The reason for this is that everything done to make 
> subsurfaces not blink or move incorrectly is just as useful for floating 
> menus and transient windows, and everything for handling events and 
> grabs for transient windows is useful for subwindows.

Surfaces and windows are different concepts. Windows do not exist in
the core, they exist in the shell. Some of the window types, like
perhaps tooltips, could be implemented as sub-surfaces instead, since
they do not have additional semantics, unlike for instance a menu which
needs(?) a grab. If there are any semantics or additional behaviour
involved, as opposed to simply just being a piece of a solid window, it
needs shell protocol and is not suitable as simply a sub-surface.

In my mind, sub-surface is a role for a surface, and so excludes any
shell specific behaviour as those are different roles. Just like the
pointer image role.

> In my design this is part of wl_shell_surface, not the wl_surface. There 
> is no difference between children and main surfaces, in fact they can 
> change from one to the other. The new request would be called set_parent 
> and it is done to the "child" wl_shell_surface. The request has the 
> following arguments: (all the binary ones would be bits in a single 
> integer):

Right. I went intentionally for the core protocol instead of shell
protocol, since I think this is a more low-level feature. It can be
shell agnostic, and useful on non-desktop shells.

Also, I want to keep it as simple as possible. No re-parenting, no
nesting, no role changes. After all, the primary aim is to offload some
of the window content compositing to the server, to allow the use of
special hardware. I will probably drop the idea of scaling for
now, too, and see how it looks like.

Our design starting points are very different, core versus shell.

> > Clipping
> 
> Subsurfaces are exactly like transient windows and thus are not clipped 
> by the parent, exactly like you propose.
> 
> > For instance, drawing window decorations in four sub-surfaces around
> > the main content would not only waste memory, but also not solve the
> > problem of GL applications, where one needs to reserve a margin for the
> > decorations. (Hello, window decoration libraries? :-)
> 
> Did you make a typo? It sounds like 4 surrounding surfaces *will* solve 
> the problems for GL applications, right?

No, the above paragraph means, that if we force sub-surfaces to be
clipped to the parent surface, it will not solve the problem with GL;
because the parent surface must be larger than the GL content, so that
decoration surfaces do not get clipped away.

If we do not clip, all is good, problem solved.

> The shell needs to pretend the window is all of these together. I think 
> this has some implications for how the current maximize and fullscreen 
> work. My preference is that the client can just tell it to act like the 
> main window is a particular size, while the buffer is a smaller 
> rectangle. I also want this to allow maximize and docking to clip off 
> "edges" but not require the client to allocate buffer space or draw the 
> clipped edges.

Yes, I haven't thought about how that could work. I hope I can find a
way to manage it by using a configure event only for the parent
surface, when the composite surface is being resized, and then leave it
up to the client to a) use a smaller parent surface than configure
suggests for, and b) adjust the sub-surfaces so that the composite
window's size becomes the suggested size. If it is about fullscreen,
the client could also just remove the decoration sub-surfaces, and go
properly fullscreen with a single wl_surface; the parent.

It feels like that should all fall into place quite nicely, but we'll
see.

The shell will probably not even know about sub-surface objects at all,
just like it does not know about pointer image surfaces, IIRC.

> > Committing changes
> 
> I think it may work that a commit on a parent is an implied commit on 
> all the children. To make a set of child surfaces all resize in unison, 
> change them all but don't call commit on any, and call commit on the 
> main window after all are updated.
> 
> An async video player process could call commit on the child surface for 
> it's changes as the frames play. It could even change the resolution by 
> changing the local transform of the surface, though it has to result in 
> the same-sized rectangle as before.
> 
> It is possible it may be very difficult for toolkits to be written to 
> defer the commits for the window pieces. In that case some of your ideas 
> may be necessary.

Yes, it is otherwise a nice idea, but does not easily work for
the independent video sink case. We probably need more information on
how these video sinks and flash plugin APIs work, whether they have any
API for synchronizing resizes, or do they just expect the window system
to notify them directly like I assume X11 does.

Or should we just design a protocol that is nice, clean, and works, and
expect clients to deal with it. That would be the Wayland way. Daniel,
Kristian?

This really needs suggestions from toolkit implementors, too.

> > Sub-surface configuration/geometry control
> > 
> > All operations about positioning and sizing sub-surfaces take place in
> > the parent surface's coordinate frame, and are limited to integer
> > pixels. No fractional pixels are allowed, and no transformations apart
> > from translation and scaling(?) are supported.
> 
> I suspect 90 degree rotations and reflections will be required to get 
> some video decoders to display on current wayland output configurations. 

We already have the 90-deg rotations and flips as a wl_surface request,
so it works also for sub-surfaces. What we do not have is scaling.

> But I see no reason not to support arbitrary transforms. If they are 
> slow then clients won't use them but the compositor already handles them 
> so this saves no code.

Hardware generally cannot scan out arbitrary transformations, so
considering the primary aim, this feels superfluous to me. And by
"arbitrary", I assume you meant to be limited to projective
transformations.


Thanks,
pq