How to implement OSD overlay in Wayland/Weston

[Pekka Paalanen]

On Tue, 19 Nov 2019 12:48:10 +0100
Guillermo Rodriguez <guillerodriguez.dev at gmail.com> wrote:

> El mar., 19 nov. 2019 a las 11:47, Pekka Paalanen
> (<ppaalanen at gmail.com>) escribió:
> >
> > On Tue, 19 Nov 2019 09:53:39 +0100
> > Guillermo Rodriguez <guillerodriguez.dev at gmail.com> wrote:
> >  
> > > Hi all,
> > >
> > > This is a follow up to my earlier email "Let pointer events pass through".
> > >
> > > I have an use case where there is one full screen application (I'll
> > > call this the "main" application) and I need to show a OSD overlay on
> > > top of it. The OSD overlay is actually based on GStreamer (think PiP)
> > > and is managed by a different process.
> > > The OSD overlay should not handle pointer events; these should pass
> > > through to the main application, which knows how to handle them.
> > >
> > > I am not sure how to approach this in Wayland/Weston. I have
> > > considered the following:
> > >
> > > 1.
> > > Use wl_surface::set_input_region so that pointer events are not
> > > delivered to the OSD overlay, but instead pass through and are handled
> > > by the main application.
> > > This "almost" works, however when the main application receives the
> > > event, Weston brings it on top, thus hiding the OSD overlay.
> > > I searched for a way to specify that a given surface should always
> > > stay "on top". This does not seem to be supported in Wayland.  

...

> The OSD is conceptually part of the fullscreen application but due to
> several reasons it is implemented as a separate process with a CLI
> that controls the OSD contents and PiP function; the main application
> communicates with the OSD process by writing commands to the OSD
> process' stdin, and reading responses from the OSD process' stdout.
> 
> There are several reasons for this approach, but the main one is that
> if there are any issues in the GStreamer side this should not crash
> the whole application. With the current approach, the main application
> can just respawn the OSD process if it dies.

I see. Then conceptually, you more or less need a custom window manager
that knows to handle your OSD specially. This is exactly the pluggable
(shell) modules that Weston supports. More on that below.

> OK, so it seems that 2 and 3 are going to be difficult due to
> architectural reasons. Let's go back to 1.

Mind that using separate independent processes makes it nearly
impossible to synchronize their content updates. This would be an issue
if you need to move or resize the OSD in sync with the main window
contents.

> I understand that my main problem is controlling the Z-order of
> top-level surfaces. If that is not possible at the application level,
> then I guess I need to mess around with Weston.
> 
> From what I see
> (https://vignatti.com/2013/03/05/ui-customization-on-wayland/), Weston
> supports the concept of plugabble shells that are in charge of
> defining "how surfaces will be mapped on the screen". I guess then
> that I would need to write a custom shell controlling Z-order of the
> surfaces. Does this make sense?
> 
> I have been looking for documentation on this (the API for custom
> Weston shells), but I cannot find anything. Can you point me in the
> right direction?

Unfortunately it's not documented even still, yet.

One approach is to fork Weston's desktop-shell/. But since your use
case is quite simple, instead of replacing the whole shell plugin, you
could write a new add-on plugin that implements only the new behaviour
you need.

You need a little protocol extension that will allow a client to send a
request that "gives a wl_surface the role of an OSD" and any parameters
you want the client to be in control of. Alternatively the plugin might
implement wlroot's layer-shell interfaces. On initialization, the
plugin creates a wl_global for the protocol extension and handles the
OSD-specific requests through that.

Examples of such custom wl_surface roles are weston_desktop.set_panel
and weston_desktop.set_background requests in Weston's
protocol/weston-desktop-shell.xml. The server implementation of these
can be found in desktop-shell/shell.c.

Since it's always-on-top, you create a new weston_layer with a position
around WESTON_LAYER_POSITION_TOP_UI or so. When the OSD wl_surface
needs to be mapped, the weston_view of it is set into that
weston_layer. That should be all, on a high level idea.

The catch here is, that you would need to get GStreamer use the OSD
role instead of the top-level window role for the wl_surface. I believe
that should be well possible in a couple of ways even: you could write
an app-sink element, or, I do believe that waylandsink should allow
using an existing wl_display and a sub-surface... or some
Wayland-enabled sink should. I'm not too familiar with Gst but I'm sure
it's doable without too much hassle somehow.

Doing the above, you are essentially making the OSD a "desktop
component", so to speak.


Thanks,
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20191119/f3d76097/attachment-0001.sig>