Handling GLContext in GstAggregator Element (Rust)

[Matthew Waters]

Hi,

In order to properly integrate with any other GStreamer OpenGL element, 
you need to retrieve the OpenGL display and context objects from 
elsewhere in the pipeline through the GstContext mechanism.

In C, this is: 
https://gstreamer.freedesktop.org/documentation/gl/gstglutils.html?gi-language=c#gst_gl_ensure_element_data 
for ensuring you have the display and the application provided context. 
https://gstreamer.freedesktop.org/documentation/gl/gstglutils.html?gi-language=c#gst_gl_handle_set_context 
for handling GstElementClass::set_context. 
https://gstreamer.freedesktop.org/documentation/gl/gstglutils.html?gi-language=c#gst_gl_handle_context_query 
for responding to queries of type GST_QUERY_CONTEXT for providing the 
display, other_context and context in use to the surrounding peers. 
https://gstreamer.freedesktop.org/documentation/gl/gstglquery.html?gi-language=c#gst_gl_query_local_gl_context 
for asking the surrounding elements for the GL context they are using.

If all that fails, then there is 
https://gstreamer.freedesktop.org/documentation/gl/gstgldisplay.html?gi-language=c#gst_gl_display_get_gl_context_for_thread 
for retrieving the GL context for a specific thread (NULL can be 
provided to get any GL context).

Ultimately however, you are reimplementing what has been implemented by 
these two classes: 
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/main/subprojects/gst-plugins-base/ext/gl/gstglbasemixer.c 
and 
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/main/subprojects/gst-plugins-base/ext/gl/gstglmixer.c 
and time would be better spent making those classes public API and then 
generating the appropriate Rust bindings.

Cheers
-Matt

On 24/11/22 21:24, Daniel Pendse via gstreamer-devel wrote:
> Hi!
>
> Working on a project to perform GL rendering based on multiple inputs 
> in GLMemory using the gstreamer rust bindings.
> From previous suggestions we decided to use the GstAggregator as a 
> base class for our element. (Rust impl exists)
> (See prev thread for more context: 
> https://lists.freedesktop.org/archives/gstreamer-devel/2022-November/080619.html)
>
> I have stumbled upon some issues with handling the GLContext when 
> creating custom shaders. When I try to create a GLSL Stage I fail with:
>
> * (gst-launch-1.0:93053): GStreamer-GL-CRITICAL **: 10:10:35.633: 
> gst_gl_context_thread_add: assertion 'GST_IS_GL_CONTEXT (context)’ failed
>
> It might seem that I fail to crate the context somehow or I don’t set 
> it properly. How should the GLContext be handled using the aggregator 
> as a base class?
> I simply create the GLContext in the start method for my element and 
> intent to pass the context trough to the methods where I intend to use 
> GL calls.
>
> // fn start
> let gl_display = GLDisplay::new();
> let gl_context = GLContext::new(&gl_display);
> gl_context.activate(true).unwrap();
>
> let shader_program = GLShader::new(gl_context);
> let vertex_shader = GLSLStage::new_default_vertex(gl_context); // Crash
>
> And also can I expect to input GLMemory and output GLMemory using this 
> baseclass  by using glupload/gldownload?
> I am guessing that there might be a performance hit if I don’t 
> input/output GLMemory, and instead handle that in the element..
> I have previous successfully managed to get gl textures passing 
> through the element like so feels like I am not handling the context 
> correct.
> (gst-launch-1.0 videotestsrc ! ... ! glupload ! <ELEMENT>  ! 
> gldownload ! ... ! autovideosink
>
> Any tips and help is greatly appreciated!
>
> Daniel Pendse

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20221125/e5245fdf/attachment.sig>