[PATCH weston v3 01/13] protocol: add sub-surfaces

Mon Apr 29 10:11:56 PDT 2013

On Mon, Apr 29, 2013 at 5:16 AM, Pekka Paalanen <ppaalanen at gmail.com> wrote:

> On Mon, 29 Apr 2013 09:40:56 +0200
> David Herrmann <dh.herrmann at gmail.com> wrote:
>
> > Hi Pekka
> >
> > On Mon, Apr 29, 2013 at 8:58 AM, Pekka Paalanen <ppaalanen at gmail.com>
> > wrote:
> > > On Fri, 26 Apr 2013 11:38:12 -0500
> > > Jason Ekstrand <jason at jlekstrand.net> wrote:
> > >
> > >> On Fri, Apr 26, 2013 at 9:14 AM, Jason Ekstrand
> > >> <jason at jlekstrand.net>wrote:
> > >>
> > >> > pq,
> > >> >
> > >> > On Fri, Apr 26, 2013 at 7:09 AM, David Herrmann
> > >> > <dh.herrmann at gmail.com>wrote:
> > >> >
> > >> >> Hi Pekka
> > >> >>
> > > ...
> > >> >> > How's this?
> > >> >> >
> > >> >>
> http://cgit.collabora.com/git/user/pq/weston.git/commit/?h=subsurface-wip2&id=0dce7236606e0a433390c970560c3050247c4e60&context=6
> > >> >>
> > >> >> Perfect, that solves all my concerns.
> > >> >>
> > >> >
> > >> > My thoughts were basically the same as David's and those changes
> > >> > solve my concerns as well.  I'm still not sure I'm 100% happy
> > >> > with how it requires you to build strange trees in certain
> > >> > cases.  That said, I think it looks pretty good and certainly
> > >> > good enough to go into weston so we can all start playing with
> > >> > it.  I'm hoping I can get sub-surfaces implemented in my java
> > >> > compositor soonish. Looking good! --Jason Ekstrand
> > >> >
> > >>
> > >> Pekka asked me to send an e-mail with a little clarification as to
> > >> what I meant by "strange trees".  Consider an EGL video surface
> > >> with SHM CSD (see attachment).  To make things more complicated,
> > >> let's say that the video surface is controlled by an external
> > >> library or (worse yet) is a foreign surface controlled by a
> > >> different process.  Due to current commit behavior and the way
> > >> commits cascade, the EGL surface (which really is the primary
> > >> surface in this case) can't be used as the primary surface.
> > >> Instead, you would have to use one of the decoration surfaces as
> > >> the parent surface (again, see the picture).
> > >
> > > Thank you, that makes your concern crystal clear.
> > >
> > >> One solution to this would be to allow some sort of surface that
> > >> contains no actual pixels but gets mapped anyway.  Right now you
> > >> can hack this with a 1x1 transparent buffer or something similar.
> > >> However, I would rather not see such hacks common place.  One way
> > >> to achieve this would be to have some sort of empty buffer that
> > >> has a size but no data.  Then you could attach one of these empty
> > >> buffers to a surface to get it mapped.
> > >
> > > I consider a special empty wl_buffer with fake size a hack, just
> > > like the 1x1 completely transparent wl_buffer, except worse,
> > > because it needs more special protocol to create.
> > >
> > >> The other advantage of an "empty buffer" is that, if it's used as
> > >> the main window, could be the full size of the window instead of
> > >> being 1x1.  This simplifies the problem of handling input across
> > >> subsurfaces as you can have the parent surface handle them.  It
> > >> would also allow us (if we wanted) to clip subsurfaces to the
> > >> parent surface without causing any major problems. Not that we
> > >> necessarily should, but it would leave open the option.
> > >
> > > The 1x1 buffer could be scaled to any size with a generic crop &
> > > scale extension, but that of course is yet another optional
> > > extension to depend on.
> > >
> > > Let's see how others feel about this, is the awkward surface
> > > structure too bad.
> > >
> > > Like I said in another email, I'm starting to think we are now
> > > beginning to accumulate hacks on top of hacks, because the original
> > > design turned out to be not flexible enough. We don't have an
> > > abstract window object in the protocol, which might have provided
> > > the sub-surface functionality in a straightforward way. A
> > > wl_window, defining what we now call the surface-local coordinate
> > > system, having no content or size on its own but attach point for
> > > an arbitrary number of wl_surfaces, being the item for role
> > > assignment.... but doing that in the shell level of the protocol
> > > instead of core would be problematic for re-usable components like
> > > video sink elements, since it does not provide nesting with
> > > wl_surfaces.
> >
> > Just as a side-note: a wl_window object doesn't really help here.
> > Assume you have an independent client library that provides two
> > sub-surfaces that ought to be displayed right next to each other (like
> > a video-screen with a video-list-sidebar). You have two choices here:
> >
> > 1) make the video-screen the parent of the sidebar
> > 2) provide two independent sub-surfaces to the caller
> >
> > Maybe the example isn't chosen well enough, but in my eyes both
> > solutions seem wrong. Instead, a dummy wl_surface that can be used by
> > the library as explicit parent would solve this case in a nice way.
> >
> > And always keep in mind that windows are not static. So in Jason's
> > case, switching the window to fullscreen causes the decorations to be
> > hidden and a client would have to de-couple the video-screen from the
> > parent and instead make it the main-window (while fullscreen).
> >
> > So a wl_window object would again require a wl_subwindow object and it
> > would basically just be a wl_surface with a transparent buffer. So why
> > not add a call to bind a dummy 100% transparent buffer to a surface.
> > This allows clients to insert invisible parent windows anywhere they
> > want so it is easier to move/change/restack sub-surfaces. And it
> > allows compositors to handle these surfaces as special surfaces to
> > avoid overhead.
>
> I am sad to realize, that even how ugly it is, dummy wl_surfaces are
> probably the best way to deal with it. To organize a tree of surfaces...
>
> For the shell, you want a wl_surface that is not really a surface, so
> you can plug and unplug decoration surfaces at will, without giving the
> primary commit control to any real surface, that might not be in your
> direct control.
>
> So, if we go on that path, could we do without dummy wl_buffers?
> Could we just remove all the mapping rules from the sub-surface
> protocol? Then the dummy surfaces would not need any wl_buffer. The
> problem is, the main surface could not be dummy, as it needs to get
> mapped by shell, which is triggered by committing a non-NULL wl_buffer.
>

Really, I think what we need to fix this problem is simply an invisible
surface.  I don't really care if this is implemented with an invisible
buffer or with some sort of "map without buffer" or "attach invisible"
request on wl_surface.  There are a lot of different ways to do it, but
they all achieve the same thing.  I don't really want to do any of this 1x1
stuff because that will make a hack a standard code path.

> I'm starting to question the starting points of this whole design:
> independent library components obviously cannot be independent due to
> the input protocol, if they ever want input. Trying to make
> application<->plugin APIs simpler by complicating the protocol.
> Arbitrary nesting with arbitrary and independent actors (components) on
> each level.
>

Yeah, we may want to just say that the main surface gets all the input
(That means it has to be full-sized, but if we have invisible surfaces,
that's not a problem).  Then we can just let the client make the input
cascade (the toolkits all do anyway).

> Maybe we should not have wl_surface.commit, or the implicit
> "roles" for wl_surfaces, that create unobvious error cases. We would
> have wl_shell_surface.commit; wl_pointer_surface.commit;
> wl_drag_surface.commit; The ability to attach an arbitrary number of
> surfaces to a wl_shell_surface, and just one to the others. No nesting,
> since input does not go to surfaces anyway, it just references them. ...
>
> Have I been solving a right problem to begin with?
>

Yes, but perhaps we should only bother solving it for visual output, not
input.  That might make it more sensible.

> Was it incorrect to assume, that the protocol can be extended piece by
> piece?
>

I actually think your "roles" concept is great. As long as we restrict to
one role per surface, it should work ok.

> Am I getting too existential? Someone hit me with a large trout. :-p
>

Yes, you are.  Done (see IRC).

Thanks,
--Jason Ekstrand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/wayland-devel/attachments/20130429/1b6db7c6/attachment.html>