Wayland debugging with Qtwayland, gstreamer waylandsink, wayland-lib and Weston

[Terry Barnaby]

Hi Pekka,

Thanks for the response. Notes below:

Terry

On 26/02/2024 13:28, Pekka Paalanen wrote:
> On Sun, 25 Feb 2024 08:04:30 +0000
> Terry Barnaby <terry1 at beam.ltd.uk> wrote:
>
>> Hi,
>>
>> I have investigated a bit further. I have built my own Weston server to
>> run under X11 on Fedora37 so I can add printf's and debug more easily
>> than using a cross compiled iMX8mp target system etc. I added a new
>> dsmc-shell to Weston which is identical to kiosk-shell (apart from
>> names) so I could play with that.
>>
>> When It run my simple QWidget test program (test016-qt6-video-example)
>> which creates one top level QWidget with a child QWidget to display the
>> Gstreamer video in it, I see the following surfaces/subsurfaces when
>> desktop_surface_committed() is called in the dsmc-shell.
>>
>> Surface: 16 1044,620 mapped: 1 opaque: 0
>>    View: 0x29182b0
>>    Surface: 18 0,0 mapped: 0 opaque: 0
>>     Surface: 44 640,480 mapped: 1 opaque: 0
>>     Surface: 18 0,0 mapped: 0 opaque: 0
>>    Surface: 17 0,0 mapped: 0 opaque: 0
>>    Surface: 16 1044,620 mapped: 1 opaque: 0
> Btw. the sub-surface list also contains the parent weston_surface in
> it, that's why surface 18 and 16 show up twice, I guess. It's used for
> z-ordering.

Yes, that was my view.


>
>> Surface 16 is used by the top level QWidget, surface 18 is used by the
>> Video QWidget and surface 44 is the GStreamer video display surface (I
>> think!). This list is generated traversing the weston_surface's views
>> and subsurface_list lists. The mapped is the "is_mapped" field of the
>> surface.
>> Only the top level weston_surface has a weston_view in the views list it
>> that is any relevance. "weston-debug scene-graph" only shows the tope
>> most surface, no subsurfaces.
> Right.
>
> A sub-surface requires its parent surface to be mapped in order to show
> up on screen. This applies recursively, so surface 18 not being mapped
> prevents surface 44 from showing up.
>
> IIRC, more or less only "fully mapped" weston_surfaces (as in, if it's
> a sub-surface, the whole sub-surface ancestry path up to a top-level is
> mapped) have a weston_view. weston_view defines where on screen a
> weston_surface will be presented, so without a view it cannot show up.

Ok. What Wayland API requests cause a surface to actually be mapped 
(Sorry don't really know Wayland protocol) ?

The higher level surfaces are created/managed by QtWayland, but maybe I 
can override somehow.


>
>> As mentioned before, If I use QPainter to draw into the video QWidget it
>> actually draws into the top QWidgets 16 surface using Wayland protocol.
>> I would have thought it would draw into its own QWidget surface 18 as it
>> has Qt::WA_NativeWindow set ?
>>
>> I assume that Qtwayland is not "activating" the video QWidget's surface
>> or using it for some reason (send subsurface expose events ?) ?
>>
> If that's true, then it is very relevant. A sub-surface becomes mapped
> and visible when:
>
> - its parent surface is mapped and visible, and
>
> - the parent surface is committed after the sub-surface has been
>    created and associated, and
>
> - if the sub-surface is in synchronized mode, there also needs to be a
>    parent surface commit after every sub-surface commit you want to
>    become visible. So if you do the first sub-surface sync-mode commit
>    with a buffer after the parent has already committed the
>    sub-surface's creation, the parent surface needs too commit again.
>
> This applies recursively, too, and you have a sub-sub-surface there.
>
> Do you actually need to sub-surface in the middle? Have you tried
> without it?

I am not doing anything with Wayland directly. Qt is managing the higher 
level surfaces/subsurfaces and then GStreamers waylandsink is passed one 
of these Qt managed surfaces and it creates the subsurface 44. Looking 
at waylandsink it should set this subsurface to be desynced so it can 
draw into this surface without synchronising to the parents surface 
managed by Qt.

All I am trying to do is use the technique as mentioned in various 
forums/lists to get GStreamer to display a video such that it "appears" 
where a QWidget is on the screen. Mind you all of this info is very 
rough and ready. For X11 it appears stable, but for Qt/Wayland the info, 
and it appears functionality, is not quite all there.

When you say a sub-surface in the middle I presume you mean the surface 
18 of the lower QWidget ? Well I have tried using the top most QWidget's 
surface 16 and the video is displayed, although all over the 
application. I really need to manage this surface so it can be raised, 
lowered and resizes amongst the other QWidgets somehow. I have tried 
using direct Wayland API calls to create a subsurface manually from the 
top surface but so far I have just got protocol errors while trying 
this. It may be my bad Wayland client code or how it is interfering with 
Qt's Wayland interface.

I have even tried using a separate top level surface. Unfortunately as 
the standard Wayland protocols do not allow an application to move or 
manage top level Windows this is not useful. I guess I could create an 
extra Wayland/Weston server protocol to allow my app (embedded system) 
to manage these windows, but I was trying to do things in as standard a 
way as possible.



>
>
> Thanks,
> pq
>
>> I note in the qtwayland change logs, for the earlier QT5 for subsurface
>> changes:
>> dist/changes-5.6.3: - [QTBUG-52118] Fixed subsurface positions sometimes
>> not being committed.
>> dist/changes-5.11.2: - [QTBUG-69643] Fixed a bug where subsurfaces would
>> not be rendered if clients added them before a WaylandQuickItem was
>> created for the parent surface
>> dist/changes-5.12.0: - [QTBUG-49809] Added support for
>> wl_subsurface.place_above and place_below in WaylandQuickItem.
>> dist/changes-5.15.2: - [QTBUG-86176] We now send subsurface expose
>> events when a different toplevel (such as a dialog) is configured.
>>
>> Could any of these be related ?
>>
>> Terry