Wayland gtk windowing (glimagesink)

Mon Apr 29 06:53:21 UTC 2024

I forgot to mention that handling window operations in Wayland is a bit
different from X11.there is a c example for wayland gtk in the gstreamer
repo, but I can not find it now.
here is my python code, shall be easy convert to C++

libgdk = ctypes.CDLL(find_library("libgdk-3"))
libgst = ctypes.CDLL(find_library("libgstreamer-1.0"))

def get_wayland_window_handle(widget):
return libgdk.gdk_wayland_window_get_wl_surface(hash(widget.get_window()))

def get_default_wayland_display_context():
wl_display = libgdk.gdk_wayland_display_get_wl_display(
hash(Gdk.Display.get_default())
)
context = Gst.Context.new("GstWaylandDisplayHandleContextType", True)
structure = libgst.gst_context_writable_structure(hash(context))
libgst.gst_structure_set(
structure,
ctypes.c_char_p("display".encode()),
hash(GObject.TYPE_POINTER),
wl_display,
0,
)
return context

def set_window(self):
if self.window is not None:
# window already set, when refresh/update, do not create new window
return
window = Gtk.Window()
window.set_title("Video Streaming")
self.window = window
self.window.connect("destroy", Gtk.main_quit, "WM destroy")
window.set_default_size(1200, 800)
self.window.show_all()
self.window.fullscreen()

def set_stream(self, url):
command = f"rtspsrc name=rtspsrc location={url} protocols=tcp latency=10
max-rtcp-rtp-time-diff=10 ! errorignore ! rtph265depay name=depay !
h265parse ! avdec_h265 name=avdec ! videoconvert ! video/x-raw, format=RGBA
! waylandsink name=sink"
self.logger.info(f"command {command}")
self.pipeline = Gst.parse_launch(command)

bus = self.pipeline.get_bus()
bus.add_signal_watch()
bus.connect("message", self.on_message)

if self.drawing_area is not None:
self.window.remove(self.drawing_area)
self.drawing_area = Gtk.DrawingArea()
# set background color
self.drawing_area.modify_bg(
Gtk.StateType.NORMAL, Gdk.color_parse(self.background_color_str)
)

glsink = self.pipeline.get_by_name("sink")

# Wayland display context wrapped as a GStreamer context.
wl_display = get_default_wayland_display_context()
glsink.set_context(wl_display)

self.window.connect("delete-event", Gtk.main_quit)
self.window.add(self.drawing_area)
self.drawing_area.realize()

self.window.show_all()

bus.enable_sync_message_emission()
bus.connect("sync-message", self.on_sync_message)

def on_sync_message(self, bus: Gst.Bus, message: Gst.Message):
if message.get_structure() is None:
return 1

if message.get_structure().get_name() == "prepare-window-handle":
self.logger.info(f"[on_sync_message] {message.get_structure().get_name()}")
sink = message.src
self.logger.info(f"[on_sync_message] sink: {sink}")

# Wayland window handle.
wl_handle = get_wayland_window_handle(self.drawing_area)
sink.set_window_handle(wl_handle)

allocation = self.drawing_area.get_allocation()
sink.set_render_rectangle(
allocation.x, allocation.y, allocation.width, allocation.height
)

On Mon, 29 Apr 2024 at 12:15, Gregoire Gentil <gregoire at gentil.com> wrote:

> Thanks for helping. Still not working.
>
> I have replaced glimagesink by waylandsink. If I don't call
> "gst_video_overlay_set_window_handle", the video appears in the back. So
> at least, waylandsink is working. But note that the video is appearing
> without being part of any window at all.
>
> If I call "gst_video_overlay_set_window_handle", the video doesn't
> appear and doesn't play. I also receive the following message:
>
> waylandsink cannot use an externally-supplied surface without an
> externally-supplied display handle. Consider providing a display handle
> from your application with GstContext
>
> Am I missing another API call?
>
> Grégoire
>
>
> On 4/28/24 20:28, Max Weng wrote:
> > I understand your situation better now. :) I've noticed that the video
> > indeed appears in a separate window with other sinks as well. It seems
> > that in my experience, only |waylandsink|consistently integrates
> > directly into the GTK window without issues.
> >
> > On Mon, 29 Apr 2024 at 10:51, Gregoire Gentil <gregoire at gentil.com
> > <mailto:gregoire at gentil.com>> wrote:
> >
> >     I could have been more precise: the glimageslink pipeline does work
> and
> >     the video appears in a separated window if I don't link it to the
> >     window
> >     with the code mentioned below. What I want to do is to insert the
> video
> >     inside the gtk window.
> >
> >     Also glimagesink has the unique advantage to work on both wayland and
> >     non-wayland systems,
> >
> >     Grégoire
> >
> >
> >
> >
> >     On 4/28/24 19:12, Max Weng wrote:
> >      > try use waylandsink?
> >      >
> >      > this work for me "rtspsrc name=rtspsrc location={url}
> protocols=tcp
> >      > latency=10 max-rtcp-rtp-time-diff=10 ! errorignore ! rtph265depay
> >      > name=depay ! h265parse ! avdec_h265 name=avdec ! videoconvert !
> >      > video/x-raw, format=RGBA ! waylandsink name=sink"
> >      >
> >      > On Mon, 29 Apr 2024 at 06:18, Gregoire Gentil via gstreamer-devel
> >      > <gstreamer-devel at lists.freedesktop.org
> >     <mailto:gstreamer-devel at lists.freedesktop.org>
> >      > <mailto:gstreamer-devel at lists.freedesktop.org
> >     <mailto:gstreamer-devel at lists.freedesktop.org>>> wrote:
> >      >
> >      >     Hello,
> >      >
> >      >     I had an application with a gtk+-3.0 window and a pipeline
> "... !
> >      >     imagesink". The video is rendered inside the gtk window.
> >      >
> >      >     I was doing:
> >      >     static void realize_cb(GtkWidget *widget, CustomData *data) {
> >      >     GdkWindow *window = gtk_widget_get_window(widget);
> >      >     data->window_handle = GDK_WINDOW_XID(window);
> >      >
> >      >     and then:
> >      >
> >      >     static GstBusSyncReply bus_sync_handler(GstBus * bus,
> >     GstMessage *
> >      >     message, CustomData *data) {
> >      >
> >
>  gst_video_overlay_set_window_handle(GST_VIDEO_OVERLAY(data->video_sink),
> >      >     data->window_handle);
> >      >
> >      >     Everything was working fine. I want now to support wayland
> >     system.
> >      >
> >      >     Pipeline becomes: "... ! glimagesink".
> >      >
> >      >     in the realize_cb function, I have now:
> >      >     GdkWindow *window = gtk_widget_get_window(widget);
> >      >     data->window_handle =
> >      >     (guintptr)gdk_wayland_window_get_wl_surface(window);
> >      >
> >      >
> >      >     At run-time, I'm getting hundreds of:
> >      >     "GStreamer-GL-CRITICAL **: 14:06:48.263: Failed to flush
> Wayland
> >      >     connection"
> >      >
> >      >     What am I doing wrong? Many thanks in advance for any hint,
> >      >
> >      >     Grégoire
> >      >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20240429/7ea75c75/attachment-0001.htm>