[PATCH weston v3 01/13] protocol: add sub-surfaces

[Pekka Paalanen]

On Mon, 29 Apr 2013 09:40:56 +0200
David Herrmann <dh.herrmann at gmail.com> wrote:

> Hi Pekka
> 
> On Mon, Apr 29, 2013 at 8:58 AM, Pekka Paalanen <ppaalanen at gmail.com>
> wrote:
> > On Fri, 26 Apr 2013 11:38:12 -0500
> > Jason Ekstrand <jason at jlekstrand.net> wrote:
> >
> >> On Fri, Apr 26, 2013 at 9:14 AM, Jason Ekstrand
> >> <jason at jlekstrand.net>wrote:
> >>
> >> > pq,
> >> >
> >> > On Fri, Apr 26, 2013 at 7:09 AM, David Herrmann
> >> > <dh.herrmann at gmail.com>wrote:
> >> >
> >> >> Hi Pekka
> >> >>
> > ...
> >> >> > How's this?
> >> >> >
> >> >> http://cgit.collabora.com/git/user/pq/weston.git/commit/?h=subsurface-wip2&id=0dce7236606e0a433390c970560c3050247c4e60&context=6
> >> >>
> >> >> Perfect, that solves all my concerns.
> >> >>
> >> >
> >> > My thoughts were basically the same as David's and those changes
> >> > solve my concerns as well.  I'm still not sure I'm 100% happy
> >> > with how it requires you to build strange trees in certain
> >> > cases.  That said, I think it looks pretty good and certainly
> >> > good enough to go into weston so we can all start playing with
> >> > it.  I'm hoping I can get sub-surfaces implemented in my java
> >> > compositor soonish. Looking good! --Jason Ekstrand
> >> >
> >>
> >> Pekka asked me to send an e-mail with a little clarification as to
> >> what I meant by "strange trees".  Consider an EGL video surface
> >> with SHM CSD (see attachment).  To make things more complicated,
> >> let's say that the video surface is controlled by an external
> >> library or (worse yet) is a foreign surface controlled by a
> >> different process.  Due to current commit behavior and the way
> >> commits cascade, the EGL surface (which really is the primary
> >> surface in this case) can't be used as the primary surface.
> >> Instead, you would have to use one of the decoration surfaces as
> >> the parent surface (again, see the picture).
> >
> > Thank you, that makes your concern crystal clear.
> >
> >> One solution to this would be to allow some sort of surface that
> >> contains no actual pixels but gets mapped anyway.  Right now you
> >> can hack this with a 1x1 transparent buffer or something similar.
> >> However, I would rather not see such hacks common place.  One way
> >> to achieve this would be to have some sort of empty buffer that
> >> has a size but no data.  Then you could attach one of these empty
> >> buffers to a surface to get it mapped.
> >
> > I consider a special empty wl_buffer with fake size a hack, just
> > like the 1x1 completely transparent wl_buffer, except worse,
> > because it needs more special protocol to create.
> >
> >> The other advantage of an "empty buffer" is that, if it's used as
> >> the main window, could be the full size of the window instead of
> >> being 1x1.  This simplifies the problem of handling input across
> >> subsurfaces as you can have the parent surface handle them.  It
> >> would also allow us (if we wanted) to clip subsurfaces to the
> >> parent surface without causing any major problems. Not that we
> >> necessarily should, but it would leave open the option.
> >
> > The 1x1 buffer could be scaled to any size with a generic crop &
> > scale extension, but that of course is yet another optional
> > extension to depend on.
> >
> > Let's see how others feel about this, is the awkward surface
> > structure too bad.
> >
> > Like I said in another email, I'm starting to think we are now
> > beginning to accumulate hacks on top of hacks, because the original
> > design turned out to be not flexible enough. We don't have an
> > abstract window object in the protocol, which might have provided
> > the sub-surface functionality in a straightforward way. A
> > wl_window, defining what we now call the surface-local coordinate
> > system, having no content or size on its own but attach point for
> > an arbitrary number of wl_surfaces, being the item for role
> > assignment.... but doing that in the shell level of the protocol
> > instead of core would be problematic for re-usable components like
> > video sink elements, since it does not provide nesting with
> > wl_surfaces.
> 
> Just as a side-note: a wl_window object doesn't really help here.
> Assume you have an independent client library that provides two
> sub-surfaces that ought to be displayed right next to each other (like
> a video-screen with a video-list-sidebar). You have two choices here:
> 
> 1) make the video-screen the parent of the sidebar
> 2) provide two independent sub-surfaces to the caller
> 
> Maybe the example isn't chosen well enough, but in my eyes both
> solutions seem wrong. Instead, a dummy wl_surface that can be used by
> the library as explicit parent would solve this case in a nice way.
> 
> And always keep in mind that windows are not static. So in Jason's
> case, switching the window to fullscreen causes the decorations to be
> hidden and a client would have to de-couple the video-screen from the
> parent and instead make it the main-window (while fullscreen).
> 
> So a wl_window object would again require a wl_subwindow object and it
> would basically just be a wl_surface with a transparent buffer. So why
> not add a call to bind a dummy 100% transparent buffer to a surface.
> This allows clients to insert invisible parent windows anywhere they
> want so it is easier to move/change/restack sub-surfaces. And it
> allows compositors to handle these surfaces as special surfaces to
> avoid overhead.

I am sad to realize, that even how ugly it is, dummy wl_surfaces are
probably the best way to deal with it. To organize a tree of surfaces...

For the shell, you want a wl_surface that is not really a surface, so
you can plug and unplug decoration surfaces at will, without giving the
primary commit control to any real surface, that might not be in your
direct control.

So, if we go on that path, could we do without dummy wl_buffers?
Could we just remove all the mapping rules from the sub-surface
protocol? Then the dummy surfaces would not need any wl_buffer. The
problem is, the main surface could not be dummy, as it needs to get
mapped by shell, which is triggered by committing a non-NULL wl_buffer.

I'm starting to question the starting points of this whole design:
independent library components obviously cannot be independent due to
the input protocol, if they ever want input. Trying to make
application<->plugin APIs simpler by complicating the protocol.
Arbitrary nesting with arbitrary and independent actors (components) on
each level.

Maybe we should not have wl_surface.commit, or the implicit
"roles" for wl_surfaces, that create unobvious error cases. We would
have wl_shell_surface.commit; wl_pointer_surface.commit;
wl_drag_surface.commit; The ability to attach an arbitrary number of
surfaces to a wl_shell_surface, and just one to the others. No nesting,
since input does not go to surfaces anyway, it just references them. ...

Have I been solving a right problem to begin with?
Was it incorrect to assume, that the protocol can be extended piece by
piece?
Am I getting too existential? Someone hit me with a large trout. :-p


- pq