[rust] waylandsink with video overlay

[Sebastian Dröge]

On Wed, 2021-01-27 at 15:54 -0800, Joel Winarske wrote:
> I am trying to add a video overlay to a wayland surface.
> 
> I'm seeing this runtime error:
> Error! Received error from /GstPipeline:pipeline0/GstWaylandSink:waylandsink0: Application did not provide a wayland display handle (debug: Some("../ext/wayland/gstwaylandsink.c(946): gst_wayland_sink_set_window_handle (): /GstPipeline:pipeline0/GstWaylandSink:waylandsink0:\nwaylandsink cannot use an externally-supplied surface without an externally-supplied display handle. Consider providing a display handle from your application with GstContext"))
> 
> What is the pattern for adding display handle with GstContext in Rust?
>
> There doesn't appear to be a reference to this in any of the samples.
> 
> I believe I have set the video_overlay window handle (pointer to wl_surface) correctly:
> https://github.com/jwinarske/waylandsink-with-video-overlay-rs/blob/main/src/main.rs#L170

Wayland is a bit special compared to all other platforms in this regard
unfortunately.

You can find the relevant code in C here:
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/blob/66eed8a61d14d08daff2d37b5ae22e8e7c8c05a7/tests/examples/waylandsink/main.c#L84-132

Basically in addition to setting the wl_surface as window handle, you
also need to set a GstContext on the pipeline (or at least the sink)
with the wl_display.

This requires using the libgstwayland library, which unfortunately is
not included in the bindings yet.

You could call those functions via Rust FFI directly after making sure
that your application links to the library, or you could replicate that
function in Rust too:

https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/blob/66eed8a61d14d08daff2d37b5ae22e8e7c8c05a7/gst-libs/gst/wayland/wayland.c#L43-51

Due to the raw pointer type used there you'll still have to make use of
some unsafe code though.

I see that you're trying exactly that here already:
https://github.com/jwinarske/waylandsink-with-video-overlay-rs/blob/main/src/main.rs#L57-L64

The missing bit would be to create a GValue from the pointer

  let mut value = glib::Value::from_type(glib::Type::Pointer);
  unsafe {
    use glib::translate::ToGlibPtrMut;
    glib_sys::g_value_set_pointer(value.to_glib_none_mut().0, handle);
  }

And then setting that in the structure of the context.

-- 
Sebastian Dröge, Centricular Ltd · https://www.centricular.com