Inter-client surface embedding

[Pekka Paalanen]

On Mon, 17 Feb 2014 22:59:11 +0000 (GMT)
Mark Thomas <mark-wayland-devel at efaref.net> wrote:

> On Mon, 17 Feb 2014, Pekka Paalanen wrote:
> 
> > On Mon, 17 Feb 2014 00:04:19 +0000 (GMT)
> > Mark Thomas <mark-wayland-devel at efaref.net> wrote:
> >
> >>    - The subsurface has separate focus from the main window surface.  For
> >> the usual use cases of embedding like this, you'd prefer the parent
> >> surface to remain focused (or at least, appear focused) while the embedded
> >> surface is being interacted with.   Not sure if this is a general feature
> >> of subsurfaces, nor what could be done about it.
> >
> > set an empty input region, and the input events will fall through the
> > surface. So you intend that the embedded client never gets input for
> > the embedded surface directly?
> 
> I think that the embedded client should still get input as normal, however 
> the surface that it's embedded within should still appear to be focussed. 
> I guess this is a shell interface question, and will probably need a shell 
> interface extension.  I'll think about this more later on when I come 
> round to writing the shell plugin that I'll undoubtedly need.
> 
> > Did you know about the earlier attempts on this? I think you should be
> > able to find some discussion or a proposal by searching the
> > wayland-devel@ archives for "foreign surface" protocol, IIRC. At that
> > time, the conclusion was to use a nested mini-compositor approach
> > instead, which is demoed in weston clients as weston-nested.
> 
> I did not.  That's quite frustrating, I could have saved myself some time. 
> I went back and looked and none of the posts mentioned "embed" or 
> "plug/socket" so that's why I didn't find them. :(
> 
> Do you know if any code came about from the foreign surface proposals?

I don't recall if it did. I assume you found some email discussions,
you could ask directly from the people proposing them.

> The nested mini-compositor example doesn't build for me as I don't have 
> working EGL, so I never even noticed it!  Reading about it the approach 
> seems to be more suited to nested application situations, e.g. a web 
> browser embedding a document viewer.
> 
> For the panel use case it seems like the wrong approach, as the embedded 
> panel objects are merely fastened to the panel like badges, rather than 
> part of the panel itself.  It seems a shame to reimplement a compositor in 
> the panel when we've already got a perfectly good compositor to use.
> 
> > I see your protocol definition lacks all documentation on how it is
> > supposed to be used and implemented. A verbal description would be nice,
> > giving an overview.
> 
> I did try to give a quick overview in the email, but it was late last 
> night and I may not have been clear.
> 
> I've pushed some doc updates to the protocol.xml file my git repo.  But
> in terms of Jonas Ådahl's proposal, my protocol works the other way round:
> 
>    A creates a main surface
>    A creates a "hole" on that surface and sets its position and size
>    A gets the uid (handle) from the server
>    A passes that uid to B via IPC
>    B creates a surface
>    B creates a "plug" on that surface with the uid it got from A
>    B receives a configure event from the server with the size of the hole
>    B creates a buffer of the correct size and renders its image to the
>      surface
> 
> > How do you handle glitch-free resizing? Sub-surfaces handle glitch-free
> > resizing by temporarily changing the sub-surface into synchronized
> > mode, assuring the sub-surface has new content in the correct new size,
> > and then atomically commits the whole tree of sub-surfaces with a
> > commit to the root wl_surface. Do you have any synchronization
> > guarantees like that? With separate processes cooperating to create a
> > single window it will be even more important than with the
> > existing sub-surfaces case, and you will need more IPC between the two
> > clients. Using client1<->client2 IPC would be more efficient than
> > client1<->server<->client2.
> 
> I don't.  Sorting out glitch-free interactive resizing is delegated to the 
> clients, although you can get pretty good glitchy resizing by B repainting 
> whenever it receives the configure event.

Except if you do not define a way to push *all* new state accross the
multiple clients to the server in an atomic way, nothing the clients
can do will guarantee glitch-free operation. You cannot rely on e.g.
two wl_surface.commits happening back-to-back, there is always a
possibility that the server will repaint between them, which means the
server uses only half of the new state, leading to a visual glitch.

It is very easy to design a protocol that works glitch-free most of the
time, and it may be hard to even purposefully trigger the potential
glitches, but the race can still be there.

> My anticipated use case is applets inside panels, which aren't typically 
> resized, so this implementation should be sufficient.

Right, but if you build a generic protocol for embedding, you really
should try hard to solve all these ugly, nasty corner cases. Experience
has shown to me that hasty design decisions done in the past will come
back haunting when you later need to extend some other functionality.
Fixing corner cases will be much much harder after a protocol extension
has been released, which is why all hard cases you can think of should
be addressed early.

That is why I suggest starting with something specific to your use
case, rather than writing a "generic" interface that works only
for your specific use case. Or in other words, call it what it is: a
protocol for embedding panel things.

> > Have you considered if your use case could be better served by
> > moving some functionality into a special DE-specific client (e.g.
> > weston-desktop-shell) and having separate protocol (an alternative
> > "shell" interface) for panel clients to tell their wl_surface needs to
> > be embeded into the panel, rather than implementing a generic
> > mechanism where you need to solve all corner-cases in a generic way?
> > If the protocol extension was designed particularly for panels, you
> > might have an easy way out by defining special resize behaviour which
> > would avoid most client<->client negotiation.
> 
> My plan was to patch Gtk3 to implement GtkPlug and GtkSocket in terms of 
> this new interface and see how far that got me, so I hadn't considered 
> that as an alternative.  I could see that working.
> 
> I'm going to see exactly how much mate-panel and its applets are wedded to 
> the idea of using GtkPlug and GtkSocket.  If I can do it with a 
> mate-desktop-shell protocol extension, with minimal additional support 
> from weston core, that would be ideal.

Remember to separate the privileged interface for mate-desktop-shell
client from the public interface for panel embedded clients,
preferrably into separate globals.


Thanks,
pq