Overlay Video on Wayland

[Scott Talbert]

On Thu, 19 Apr 2018, Nicolas Dufresne wrote:

>>>>>> Is it possible to use video overlays (either with GstPlayer
>>>>>> or playbin) on
>>>>>> Wayland surfaces?  And if so, what is the recommended
>>>>>> approach?  I saw
>>>>>> there was 'waylandsink' but it doesn't seem to be public.
>>>>>
>>>>> waylansink is part of gst-plugins-bad (unstable), and can be
>>>>> used with
>>>>> GstVideoOverlay interface on wayland. Run "gst-inspect-1.0
>>>>> waylandsink"
>>>>> for details. An example application can be found here:
>>>>>
>>>>> https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tes
>>>>> ts/examples/waylandsink
>>>>
>>>> The problem, though, is that gst/wayland/wayland.h header is not
>>>> public.
>>>>
>>>> See
>>>> https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst-l
>>>> ibs/gst/wayland/Makefile.am#n26
>>>
>>> You don't really need that library. The geometry stuff can be
>>> ignored,
>>> and then the rest is just helpers around string comparison:
>>>
>>> https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst-lib
>>> s/gst/wayland/wayland.c
>>>
>>> If you really want these helpers, you can copy them in your app.
>>
>> OK.  I'm seeing a few issues with the sample application.
>>
>> First, these warnings are issued during startup (on Fedora 27):
>>
>> ** (gst-wayland-gtk-demo:11908): WARNING **: Wayland compositor is
>> missing
>> the ability to scale, video display may not work properly.
>
> Yes, I've added that warning to try and convince people that a GL
> compositer should support scaling. waylandsink without that in the
> compositor is pretty useless, and that's why waylandsink is pretty
> useless on Gnome Shell.

Well, the good news is that someone is working on adding wp_viewport 
support into mutter.  When testing those patches, waylandsink support 
seems to mostly work.

https://gitlab.gnome.org/GNOME/mutter/issues/132


>> ** (gst-wayland-gtk-demo:11908): WARNING **: Could not bind to
>> zwp_linux_dmabuf_v1
>>
>> On Debian Unstable, I see the first warning only.
>
> I have go the example builds, do you also get that warning in:
>
>  gst-launch-1.0 videotestsrc ! waylandsink
>
> I have Fedora 27, and on Gnome Shell I don't get this warning (maybe
> I'll get this problem tomorrow when I update ...).

Still getting this warning, even with Weston, so not sure what to do about 
it?

>> Second, the window seems to be drawn in the wrong location.  The
>> window
>> with the blue background and bouncing ball is shifted down and to
>> the
>> right about 100 pixels of where it should be.  This doesn't seem to
>> be a
>> problem with the coordinates passed to
>> gst_video_overlay_set_render_rectangle().  If I mess with those
>> coordinates, I can see that the black window changes position, but
>> the
>> blue window does not.

The offset problem goes away when using the wp_viewport patch, so that is 
related to the lack of wp_viewport support.

>> Thirdly, if I use the GTK+ close button to close the window, gnome-
>> shell
>> crashes.  While this is probably a gnome-shell bug, it is very
>> annoying.
>> :)

This has been fixed in mutter 3.28.2.  :)

Scott