Handling GLContext in GstAggregator Element (Rust)

[Daniel Pendse]

Hi Matt,

Thanks for a great answer, appreciate it. We got it working by retrieving
the gl context for a specific thread per your proposals.
Right now it works because we already use glupload prior in the pipeline,
but will need to handle the context more properly i.e. create one if it
doesn't exist.

Yeah I am aware of those classes, which absolutely would have been
very helpful to use instead.
Also new to the gstreamer community but definitely would like to contribute
at some point.

Thanks again,
Daniel

On Fri, 25 Nov 2022 at 04:20, Matthew Waters <ystreet00 at gmail.com> wrote:

> Hi,
>
> In order to properly integrate with any other GStreamer OpenGL element,
> you need to retrieve the OpenGL display and context objects from
> elsewhere in the pipeline through the GstContext mechanism.
>
> In C, this is:
>
> https://gstreamer.freedesktop.org/documentation/gl/gstglutils.html?gi-language=c#gst_gl_ensure_element_data
> for ensuring you have the display and the application provided context.
>
> https://gstreamer.freedesktop.org/documentation/gl/gstglutils.html?gi-language=c#gst_gl_handle_set_context
> for handling GstElementClass::set_context.
>
> https://gstreamer.freedesktop.org/documentation/gl/gstglutils.html?gi-language=c#gst_gl_handle_context_query
> for responding to queries of type GST_QUERY_CONTEXT for providing the
> display, other_context and context in use to the surrounding peers.
>
> https://gstreamer.freedesktop.org/documentation/gl/gstglquery.html?gi-language=c#gst_gl_query_local_gl_context
> for asking the surrounding elements for the GL context they are using.
>
> If all that fails, then there is
>
> https://gstreamer.freedesktop.org/documentation/gl/gstgldisplay.html?gi-language=c#gst_gl_display_get_gl_context_for_thread
> for retrieving the GL context for a specific thread (NULL can be
> provided to get any GL context).
>
> Ultimately however, you are reimplementing what has been implemented by
> these two classes:
>
> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/main/subprojects/gst-plugins-base/ext/gl/gstglbasemixer.c
> and
>
> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/main/subprojects/gst-plugins-base/ext/gl/gstglmixer.c
> and time would be better spent making those classes public API and then
> generating the appropriate Rust bindings.
>
> Cheers
> -Matt
>
> On 24/11/22 21:24, Daniel Pendse via gstreamer-devel wrote:
> > Hi!
> >
> > Working on a project to perform GL rendering based on multiple inputs
> > in GLMemory using the gstreamer rust bindings.
> > From previous suggestions we decided to use the GstAggregator as a
> > base class for our element. (Rust impl exists)
> > (See prev thread for more context:
> >
> https://lists.freedesktop.org/archives/gstreamer-devel/2022-November/080619.html
> )
> >
> > I have stumbled upon some issues with handling the GLContext when
> > creating custom shaders. When I try to create a GLSL Stage I fail with:
> >
> > * (gst-launch-1.0:93053): GStreamer-GL-CRITICAL **: 10:10:35.633:
> > gst_gl_context_thread_add: assertion 'GST_IS_GL_CONTEXT (context)’ failed
> >
> > It might seem that I fail to crate the context somehow or I don’t set
> > it properly. How should the GLContext be handled using the aggregator
> > as a base class?
> > I simply create the GLContext in the start method for my element and
> > intent to pass the context trough to the methods where I intend to use
> > GL calls.
> >
> > // fn start
> > let gl_display = GLDisplay::new();
> > let gl_context = GLContext::new(&gl_display);
> > gl_context.activate(true).unwrap();
> >
> > let shader_program = GLShader::new(gl_context);
> > let vertex_shader = GLSLStage::new_default_vertex(gl_context); // Crash
> >
> > And also can I expect to input GLMemory and output GLMemory using this
> > baseclass  by using glupload/gldownload?
> > I am guessing that there might be a performance hit if I don’t
> > input/output GLMemory, and instead handle that in the element..
> > I have previous successfully managed to get gl textures passing
> > through the element like so feels like I am not handling the context
> > correct.
> > (gst-launch-1.0 videotestsrc ! ... ! glupload ! <ELEMENT>  !
> > gldownload ! ... ! autovideosink
> >
> > Any tips and help is greatly appreciated!
> >
> > Daniel Pendse
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20221206/12df33e5/attachment.htm>