Wayland debugging with Qtwayland, gstreamer waylandsink, wayland-lib and Weston

[Marius Vlad]

Hi,
On Thu, Feb 22, 2024 at 03:21:01PM +0000, Terry Barnaby wrote:
> Hi,
> 
> We are developing a video processing system that runs on an NXP imx8
> processor using a Yocto embedded Linux system that has Qt6, GStreamer,
> Wayland and Weston.
> 
> We are having a problem displaying the video stream from GStreamer on a
> QWidget. In the past we had this working with Qt5 and older GStreamer,
> Wayland and Weston.
> 
> A simple test program also shows the issue on Fedora37 with QT6 and
> KDE/Plasma/Wayland.
I'm tempted to say if this happens on a desktop with the same Qt version and
other compositors to be an issue with Qt rather than waylandsink or
the compositor. Note that on NXP they have their own modified Weston version.
> 
> The technique we are using is to get the Wayland surface from the QWidget is
> using (It has been configured to use a Qt::WA_NativeWindow) and pass this to
> the GStreamer's waylandsink which should then update this surface with video
> frames (via hardware). This works when the QWidget is a top level Window
> widget (QWidget(0)), but if this QWidget is below others in the hierarchy no
> video is seen and the gstreamer pipeline line is stalled.
So the assumption is that aren't there other widgets which obscures this
one, when you move it below others?
> 
> It appears that waylandsink does:
> 
> Creates a surface callback:
> 
>   callback = wl_surface_frame (surface);
> 
>   wl_callback_add_listener (callback, &frame_callback_listener, self);
> 
> Then adds a buffer to a surface:
> 
>     gst_wl_buffer_attach (buffer, priv->video_surface_wrapper);
>     wl_surface_set_buffer_scale (priv->video_surface_wrapper, priv->scale);
>     wl_surface_damage_buffer (priv->video_surface_wrapper, 0, 0, G_MAXINT32,
> G_MAXINT32);
>     wl_surface_commit (priv->video_surface_wrapper);
> 
> But never gets a callback and just sits in a loop awaiting that callback.
> 
> I assume that the surface waylandsink is using, which is created using the
> original QWidget surface (sub-surface ? with window ?) is not "active" for
> some reason.
Possibly when QWidget is below in hierarcy to be a child of of a parent, 
as described in https://wayland.app/protocols/xdg-shell#xdg_toplevel:request:set_parent,
so I assume to have a different surface than the parent one. This would
be easy to determine with WAYLAND_DEBUG. Seems unlikely to a itself a 
sub-surface of a surface.
> 
> 
> I am trying to debug this, but this graphics stack is quite complicated with
> waylandsink, qtwayland, wayland-lib and Weston not to mention the NXP
> hardware levels. My thoughts are that it is something qtwayland is doing
> with the surface stack or thread locking issues (gstreamer uses separate
> threads). I also don't understand Wayland or Weston in detail. So some
> questions:
> 
> 1. Anyone seen something like this ?
Someone else reported something similar but that by causing damage, 
or moving pointer to make the video sub-surface to show up: 
https://gitlab.freedesktop.org/wayland/weston/-/issues/843.
> 
> 2. Anyone any idea one where to look ?
> 
> 3. Given the wl_surface in the Qt app or in waylandsink is there a way I can
> print out its state and the surface hierarchy easily ?
In Weston there's something called scene-graph. You can grab it by
starting Weston with with the --debug argument, then you can print 
with `weston-debug scene-graph` command. A more recent Weston version
would indent sub-surfaces by their (main) surface parent.
> 
> 4. Any idea on any debug methods to use ?
WAYLAND_DEBUG=1 as env variable.
> 
> Cheers
> 
> Terry
> 
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20240222/6d1fca6c/attachment.sig>