GStreamer video display into a Qt6 QWidget

[Terry Barnaby]

I have managed to get something working to a degree having created a 
simpler test example.
My initial issue was that the gstreamer wayland API had changed, and my 
code was not setting the GST_WAYLAND_DISPLAY_HANDLE_CONTEXT_TYPE.

I can now see gstreamer waylandsink video within a QWidget. However it 
only works if that QWidget is a top level one (created with new 
QWidget(0) ). If it has a parent widget, even just another QWidget, the 
video is not seen and appears to have hung (no error messages, no CPU 
usage).

When testing on my embedded iMX8mp platform that is using 
gstreamer1.0-plugins-bad/1.22.5.imx-r0, Qt 6.5.0 with Weston the 
gstreamer pipeline just hangs. It hangs in 
gstwaylandsink.c/gst_wayland_sink_show_frame:

   while (self->redraw_pending)
     g_cond_wait (&self->redraw_wait, &self->render_lock);

If I test on Fedora37, using Plasma(Wayland) again no video is seen, but 
with this environment I get lots of QOS warnings:
Video::gstBusCallback: type: 01000000 data: 0x2320420
  Video::gstBusCallback: QOS warning: total dropped: 180 (230.769231%)
Video::gstBusCallback: type: 01000000 data: 0x2320420
  Video::gstBusCallback: QOS warning: total dropped: 181 (226.250000%)
Video::gstBusCallback: type: 01000000 data: 0x2320420
  Video::gstBusCallback: QOS warning: total dropped: 182 (227.500000%)

All works fine under X11.

Any ideas ?

Terry

On 18/02/2024 09:24, Terry Barnaby via gstreamer-devel wrote:
> Hi,
>
> Does anyone have any pointers on how I can display a gstreamer video 
> pipelines output into a Qt6 QWidget these days ?
>
> We are programming a system on a NXP iMX8mp Yocto embedded platform. 
> This has Wayland/Weston with Qt6 and uses low level iMX8mp 2D/3D 
> hardware engines.
>
> In the past we were using Qt5 and managed to use waylandsink setting 
> the surface to draw into from a QWidget's native Wayland surface and 
> resizing the waylandsink overlay to match the QWidget size. Basically 
> use was made of gst_video_overlay_set_window_handle() and 
> gst_video_overlay_set_render_rectangle() and used the Qt5 
> QPlatformNativeInterface API to get the QWidget's Wayland surface. 
> This worked reasonably well (a bit dodgy with Qt5 Wayland support, 
> Weston/Wayland issues, and IMO messy) but it worked and was efficient.
>
> However with the latest NXP Yocto including Qt6 this does not work 
> correctly. Although the code (and a simple test case) runs and the 
> video appears on the screen, it is not actually within the QWidget. If 
> I build the same test example on Fedora, the application jut fails 
> with lots of waylandsink errors printed.
>
> Is there a better and simple way of getting a gstreamer pipe line to 
> display video in a QWidget these days ? I have seen there is the Qt 
> QML based QMultimedia system, but that adds all of the complexity of 
> QML and I have heard is quite inefficient and no longer supports a 
> manual program controlled gstreamer pipeline that we need for our 
> application.
>
> Terry
>