Wayland gtk windowing (glimagesink)
Max Weng
zhiyong.weng at gmail.com
Mon Apr 29 23:23:39 UTC 2024
Did you call set_window_handle(wl_handle) when on_sync_message triggered?
On Tue, 30 Apr 2024 at 03:04, Gregoire Gentil <gregoire at gentil.com> wrote:
> Hello Max,
>
> Based on your python code, I have added:
>
> GdkDisplay *display = gdk_display_get_default();
> struct wl_display *wl_handle = gdk_wayland_display_get_wl_display(display);
> GstContext *context =
> gst_context_new("GstWaylandDisplayHandleContextType", True);
> GstStructure *structure = gst_context_writable_structure(context);
> gst_structure_set(structure, "display", G_TYPE_POINTER, wl_handle, NULL);
> gst_element_set_context(data.video_sink, context);
>
> That's the only difference I see. But I'm still receiving the error
> message:
> Error received from element waylandsink0: Application did not provide a
> wayland display handle
> Debugging information: ../ext/wayland/gstwaylandsink.c(1102):
> gst_wayland_sink_set_window_handle ():
> /GstPipeline:pipeline0/GstWaylandSink:waylandsink0:
> waylandsink cannot use an externally-supplied surface without an
> externally-supplied display handle. Consider providing a display handle
> from your application with GstContext
>
> Any idea what I'm missing else?
>
> Grégoire
>
>
> On 4/28/24 23:53, Max Weng wrote:
> > I forgot to mention that handling window operations in Wayland is a bit
> > different from X11.there is a c example for wayland gtk in the gstreamer
> > repo, but I can not find it now.
> > here is my python code, shall be easy convert to C++
> >
> > libgdk = ctypes.CDLL(find_library("libgdk-3"))
> > libgst = ctypes.CDLL(find_library("libgstreamer-1.0"))
> >
> > def get_wayland_window_handle(widget):
> > return
> libgdk.gdk_wayland_window_get_wl_surface(hash(widget.get_window()))
> >
> >
> > def get_default_wayland_display_context():
> > wl_display = libgdk.gdk_wayland_display_get_wl_display(
> > hash(Gdk.Display.get_default())
> > )
> > context = Gst.Context.new("GstWaylandDisplayHandleContextType", True)
> > structure = libgst.gst_context_writable_structure(hash(context))
> > libgst.gst_structure_set(
> > structure,
> > ctypes.c_char_p("display".encode()),
> > hash(GObject.TYPE_POINTER),
> > wl_display,
> > 0,
> > )
> > return context
> >
> > def set_window(self):
> > if self.window is not None:
> > # window already set, when refresh/update, do not create new window
> > return
> > window = Gtk.Window()
> > window.set_title("Video Streaming")
> > self.window = window
> > self.window.connect("destroy", Gtk.main_quit, "WM destroy")
> > window.set_default_size(1200, 800)
> > self.window.show_all()
> > self.window.fullscreen()
> >
> > def set_stream(self, url):
> > command = f"rtspsrc name=rtspsrc location={url}protocols=tcp latency=10
> > max-rtcp-rtp-time-diff=10 ! errorignore ! rtph265depay name=depay !
> > h265parse ! avdec_h265 name=avdec ! videoconvert ! video/x-raw,
> > format=RGBA ! waylandsink name=sink"
> > self.logger.info(f"command {command}")
> > self.pipeline = Gst.parse_launch(command)
> >
> > bus = self.pipeline.get_bus()
> > bus.add_signal_watch()
> > bus.connect("message", self.on_message)
> >
> > if self.drawing_area is not None:
> > self.window.remove(self.drawing_area)
> > self.drawing_area = Gtk.DrawingArea()
> > # set background color
> > self.drawing_area.modify_bg(
> > Gtk.StateType.NORMAL, Gdk.color_parse(self.background_color_str)
> > )
> >
> > glsink = self.pipeline.get_by_name("sink")
> >
> > # Wayland display context wrapped as a GStreamer context.
> > wl_display = get_default_wayland_display_context()
> > glsink.set_context(wl_display)
> >
> > self.window.connect("delete-event", Gtk.main_quit)
> > self.window.add(self.drawing_area)
> > self.drawing_area.realize()
> >
> > self.window.show_all()
> >
> > bus.enable_sync_message_emission()
> > bus.connect("sync-message", self.on_sync_message)
> >
> > def on_sync_message(self, bus: Gst.Bus, message: Gst.Message):
> > if message.get_structure() is None:
> > return 1
> >
> > if message.get_structure().get_name() == "prepare-window-handle":
> > self.logger.info(f"[on_sync_message]
> {message.get_structure().get_name()}")
> > sink = message.src
> > self.logger.info(f"[on_sync_message] sink: {sink}")
> >
> > # Wayland window handle.
> > wl_handle = get_wayland_window_handle(self.drawing_area)
> > sink.set_window_handle(wl_handle)
> >
> > allocation = self.drawing_area.get_allocation()
> > sink.set_render_rectangle(
> > allocation.x, allocation.y, allocation.width, allocation.height
> > )
> >
> > On Mon, 29 Apr 2024 at 12:15, Gregoire Gentil <gregoire at gentil.com
> > <mailto:gregoire at gentil.com>> wrote:
> >
> > Thanks for helping. Still not working.
> >
> > I have replaced glimagesink by waylandsink. If I don't call
> > "gst_video_overlay_set_window_handle", the video appears in the
> > back. So
> > at least, waylandsink is working. But note that the video is
> appearing
> > without being part of any window at all.
> >
> > If I call "gst_video_overlay_set_window_handle", the video doesn't
> > appear and doesn't play. I also receive the following message:
> >
> > waylandsink cannot use an externally-supplied surface without an
> > externally-supplied display handle. Consider providing a display
> handle
> > from your application with GstContext
> >
> > Am I missing another API call?
> >
> > Grégoire
> >
> >
> > On 4/28/24 20:28, Max Weng wrote:
> > > I understand your situation better now. :) I've noticed that the
> > video
> > > indeed appears in a separate window with other sinks as well. It
> > seems
> > > that in my experience, only |waylandsink|consistently integrates
> > > directly into the GTK window without issues.
> > >
> > > On Mon, 29 Apr 2024 at 10:51, Gregoire Gentil
> > <gregoire at gentil.com <mailto:gregoire at gentil.com>
> > > <mailto:gregoire at gentil.com <mailto:gregoire at gentil.com>>> wrote:
> > >
> > > I could have been more precise: the glimageslink pipeline
> > does work and
> > > the video appears in a separated window if I don't link it to
> the
> > > window
> > > with the code mentioned below. What I want to do is to insert
> > the video
> > > inside the gtk window.
> > >
> > > Also glimagesink has the unique advantage to work on both
> > wayland and
> > > non-wayland systems,
> > >
> > > Grégoire
> > >
> > >
> > >
> > >
> > > On 4/28/24 19:12, Max Weng wrote:
> > > > try use waylandsink?
> > > >
> > > > this work for me "rtspsrc name=rtspsrc location={url}
> > protocols=tcp
> > > > latency=10 max-rtcp-rtp-time-diff=10 ! errorignore !
> > rtph265depay
> > > > name=depay ! h265parse ! avdec_h265 name=avdec !
> > videoconvert !
> > > > video/x-raw, format=RGBA ! waylandsink name=sink"
> > > >
> > > > On Mon, 29 Apr 2024 at 06:18, Gregoire Gentil via
> > gstreamer-devel
> > > > <gstreamer-devel at lists.freedesktop.org
> > <mailto:gstreamer-devel at lists.freedesktop.org>
> > > <mailto:gstreamer-devel at lists.freedesktop.org
> > <mailto:gstreamer-devel at lists.freedesktop.org>>
> > > > <mailto:gstreamer-devel at lists.freedesktop.org
> > <mailto:gstreamer-devel at lists.freedesktop.org>
> > > <mailto:gstreamer-devel at lists.freedesktop.org
> > <mailto:gstreamer-devel at lists.freedesktop.org>>>> wrote:
> > > >
> > > > Hello,
> > > >
> > > > I had an application with a gtk+-3.0 window and a
> > pipeline "... !
> > > > imagesink". The video is rendered inside the gtk
> window.
> > > >
> > > > I was doing:
> > > > static void realize_cb(GtkWidget *widget, CustomData
> > *data) {
> > > > GdkWindow *window = gtk_widget_get_window(widget);
> > > > data->window_handle = GDK_WINDOW_XID(window);
> > > >
> > > > and then:
> > > >
> > > > static GstBusSyncReply bus_sync_handler(GstBus * bus,
> > > GstMessage *
> > > > message, CustomData *data) {
> > > >
> > >
> >
> gst_video_overlay_set_window_handle(GST_VIDEO_OVERLAY(data->video_sink),
> > > > data->window_handle);
> > > >
> > > > Everything was working fine. I want now to support
> wayland
> > > system.
> > > >
> > > > Pipeline becomes: "... ! glimagesink".
> > > >
> > > > in the realize_cb function, I have now:
> > > > GdkWindow *window = gtk_widget_get_window(widget);
> > > > data->window_handle =
> > > > (guintptr)gdk_wayland_window_get_wl_surface(window);
> > > >
> > > >
> > > > At run-time, I'm getting hundreds of:
> > > > "GStreamer-GL-CRITICAL **: 14:06:48.263: Failed to
> > flush Wayland
> > > > connection"
> > > >
> > > > What am I doing wrong? Many thanks in advance for any
> > hint,
> > > >
> > > > Grégoire
> > > >
> > >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20240430/b9a85467/attachment-0001.htm>
More information about the gstreamer-devel
mailing list