Wayland gtk windowing (glimagesink)
Max Weng
zhiyong.weng at gmail.com
Mon Apr 29 06:53:21 UTC 2024
I forgot to mention that handling window operations in Wayland is a bit
different from X11.there is a c example for wayland gtk in the gstreamer
repo, but I can not find it now.
here is my python code, shall be easy convert to C++
libgdk = ctypes.CDLL(find_library("libgdk-3"))
libgst = ctypes.CDLL(find_library("libgstreamer-1.0"))
def get_wayland_window_handle(widget):
return libgdk.gdk_wayland_window_get_wl_surface(hash(widget.get_window()))
def get_default_wayland_display_context():
wl_display = libgdk.gdk_wayland_display_get_wl_display(
hash(Gdk.Display.get_default())
)
context = Gst.Context.new("GstWaylandDisplayHandleContextType", True)
structure = libgst.gst_context_writable_structure(hash(context))
libgst.gst_structure_set(
structure,
ctypes.c_char_p("display".encode()),
hash(GObject.TYPE_POINTER),
wl_display,
0,
)
return context
def set_window(self):
if self.window is not None:
# window already set, when refresh/update, do not create new window
return
window = Gtk.Window()
window.set_title("Video Streaming")
self.window = window
self.window.connect("destroy", Gtk.main_quit, "WM destroy")
window.set_default_size(1200, 800)
self.window.show_all()
self.window.fullscreen()
def set_stream(self, url):
command = f"rtspsrc name=rtspsrc location={url} protocols=tcp latency=10
max-rtcp-rtp-time-diff=10 ! errorignore ! rtph265depay name=depay !
h265parse ! avdec_h265 name=avdec ! videoconvert ! video/x-raw, format=RGBA
! waylandsink name=sink"
self.logger.info(f"command {command}")
self.pipeline = Gst.parse_launch(command)
bus = self.pipeline.get_bus()
bus.add_signal_watch()
bus.connect("message", self.on_message)
if self.drawing_area is not None:
self.window.remove(self.drawing_area)
self.drawing_area = Gtk.DrawingArea()
# set background color
self.drawing_area.modify_bg(
Gtk.StateType.NORMAL, Gdk.color_parse(self.background_color_str)
)
glsink = self.pipeline.get_by_name("sink")
# Wayland display context wrapped as a GStreamer context.
wl_display = get_default_wayland_display_context()
glsink.set_context(wl_display)
self.window.connect("delete-event", Gtk.main_quit)
self.window.add(self.drawing_area)
self.drawing_area.realize()
self.window.show_all()
bus.enable_sync_message_emission()
bus.connect("sync-message", self.on_sync_message)
def on_sync_message(self, bus: Gst.Bus, message: Gst.Message):
if message.get_structure() is None:
return 1
if message.get_structure().get_name() == "prepare-window-handle":
self.logger.info(f"[on_sync_message] {message.get_structure().get_name()}")
sink = message.src
self.logger.info(f"[on_sync_message] sink: {sink}")
# Wayland window handle.
wl_handle = get_wayland_window_handle(self.drawing_area)
sink.set_window_handle(wl_handle)
allocation = self.drawing_area.get_allocation()
sink.set_render_rectangle(
allocation.x, allocation.y, allocation.width, allocation.height
)
On Mon, 29 Apr 2024 at 12:15, Gregoire Gentil <gregoire at gentil.com> wrote:
> Thanks for helping. Still not working.
>
> I have replaced glimagesink by waylandsink. If I don't call
> "gst_video_overlay_set_window_handle", the video appears in the back. So
> at least, waylandsink is working. But note that the video is appearing
> without being part of any window at all.
>
> If I call "gst_video_overlay_set_window_handle", the video doesn't
> appear and doesn't play. I also receive the following message:
>
> waylandsink cannot use an externally-supplied surface without an
> externally-supplied display handle. Consider providing a display handle
> from your application with GstContext
>
> Am I missing another API call?
>
> Grégoire
>
>
> On 4/28/24 20:28, Max Weng wrote:
> > I understand your situation better now. :) I've noticed that the video
> > indeed appears in a separate window with other sinks as well. It seems
> > that in my experience, only |waylandsink|consistently integrates
> > directly into the GTK window without issues.
> >
> > On Mon, 29 Apr 2024 at 10:51, Gregoire Gentil <gregoire at gentil.com
> > <mailto:gregoire at gentil.com>> wrote:
> >
> > I could have been more precise: the glimageslink pipeline does work
> and
> > the video appears in a separated window if I don't link it to the
> > window
> > with the code mentioned below. What I want to do is to insert the
> video
> > inside the gtk window.
> >
> > Also glimagesink has the unique advantage to work on both wayland and
> > non-wayland systems,
> >
> > Grégoire
> >
> >
> >
> >
> > On 4/28/24 19:12, Max Weng wrote:
> > > try use waylandsink?
> > >
> > > this work for me "rtspsrc name=rtspsrc location={url}
> protocols=tcp
> > > latency=10 max-rtcp-rtp-time-diff=10 ! errorignore ! rtph265depay
> > > name=depay ! h265parse ! avdec_h265 name=avdec ! videoconvert !
> > > video/x-raw, format=RGBA ! waylandsink name=sink"
> > >
> > > On Mon, 29 Apr 2024 at 06:18, Gregoire Gentil via gstreamer-devel
> > > <gstreamer-devel at lists.freedesktop.org
> > <mailto:gstreamer-devel at lists.freedesktop.org>
> > > <mailto:gstreamer-devel at lists.freedesktop.org
> > <mailto:gstreamer-devel at lists.freedesktop.org>>> wrote:
> > >
> > > Hello,
> > >
> > > I had an application with a gtk+-3.0 window and a pipeline
> "... !
> > > imagesink". The video is rendered inside the gtk window.
> > >
> > > I was doing:
> > > static void realize_cb(GtkWidget *widget, CustomData *data) {
> > > GdkWindow *window = gtk_widget_get_window(widget);
> > > data->window_handle = GDK_WINDOW_XID(window);
> > >
> > > and then:
> > >
> > > static GstBusSyncReply bus_sync_handler(GstBus * bus,
> > GstMessage *
> > > message, CustomData *data) {
> > >
> >
> gst_video_overlay_set_window_handle(GST_VIDEO_OVERLAY(data->video_sink),
> > > data->window_handle);
> > >
> > > Everything was working fine. I want now to support wayland
> > system.
> > >
> > > Pipeline becomes: "... ! glimagesink".
> > >
> > > in the realize_cb function, I have now:
> > > GdkWindow *window = gtk_widget_get_window(widget);
> > > data->window_handle =
> > > (guintptr)gdk_wayland_window_get_wl_surface(window);
> > >
> > >
> > > At run-time, I'm getting hundreds of:
> > > "GStreamer-GL-CRITICAL **: 14:06:48.263: Failed to flush
> Wayland
> > > connection"
> > >
> > > What am I doing wrong? Many thanks in advance for any hint,
> > >
> > > Grégoire
> > >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20240429/7ea75c75/attachment-0001.htm>
More information about the gstreamer-devel
mailing list