push opengl texture to gstreamer pipeline

[Matthew Waters]

So, I think you almost have the correct sequence.

Response inline.

On 5/8/21 1:04 am, Lusine Hayrapetyan via gstreamer-devel wrote:
> Hi Folks,
> I'm struggling with the following issue and can't understand what I'm 
> doing wrong.
> I need to pass opengl texture to the gstreamer pipeline.
> I have a rendering thread where I create opengl texture, the following 
> objects created in this thread:
> /EGLDisplay display;/
> /EGLContext context;/
> /
> /
> I create gstreamer pipeline in the main thread and as described in the 
> following article sharing an X11 display and GstGLContext with the bus 
> callback.
> http://ystreet00.blogspot.com/2015/09/gstreamer-16-and-opengl-contexts.html 
> <http://ystreet00.blogspot.com/2015/09/gstreamer-16-and-opengl-contexts.html>
>
> GstGLDisplayEGL and GstGLContext are created in this way:
> /GstGLDisplayEGL* gst_display = 
> gst_gl_display_egl_new_with_egl_display (display);
> /
> /GstGLContext *gl_context =
>       gst_gl_context_new_wrapped (GST_GL_DISPLAY (gst_display),
>       (guintptr) context, GST_GL_PLATFORM_EGL, GST_GL_API_GLES2);/
>
> The first element of my pipeline is appsrc:
> /appsrc stream-type=0 emit-signals=1 format=3 
> caps=video/x-raw(memory:GLMemory),  width=300, height=300, 
> framerate=(fraction)20/1, format=(string)RGBA ! gldownload ! .../
>
> I use need-data callback to create a buffer from texture_id and and 
> push it in the appsrc:
> /g_signal_connect (state->appsrc, "need-data", G_CALLBACK (/ pushFrame 
> /), state);/
> /
> /
> /bool pushFrame()/
> /{
>      // Wrap the texture into GstGLMemory
>     GstVideoInfo vinfo;
>     gst_video_info_set_format(&vinfo, GST_VIDEO_FORMAT_RGBA, 300, 300);
>
>     GstAllocator* allocator = 
> GST_ALLOCATOR(gst_gl_memory_allocator_get_default(gl_context));
>
>     GstGLVideoAllocationParams* params = 
> gst_gl_video_allocation_params_new_wrapped_texture(
>        state->gl_context, NULL, &vinfo, 0, NULL, 
> GST_GL_TEXTURE_TARGET_2D, GST_GL_RGBA, / texture_id /,
>       NULL, 0);
> /

The use of state->gl_context is probably your OpenGL context that has 
been wrapped from the application.  This 'wrapped' OpenGL context has 
some limitations, one being that GStreamer cannot actually do a complete 
gst_gl_context_thread_add where the request is marshalled to an 
OpenGL-specific thread.  This is what that critical is complaining about 
effectively.

To do this properly, you would need to do one of two things:
1. Retrieve the OpenGL context from the downstream gldownload element 
using either the 'context' property or using an appropriate GST_CONTEXT 
QUERY or the helper gst_gl_query_local_gl_context().
2. Create your own GStreamer OpenGL context and add it to the 
GstGLDisplay using something like: 
https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/gst-libs/gst/gl/gstglbasefilter.c#L550-565 
<https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/gst-libs/gst/gl/gstglbasefilter.c#L550-565>.

Cheers
-Matt

> /
>     // The following line produces an error!!!
>     GstGLMemory* glMemory = GST_GL_MEMORY_CAST(gst_gl_base_memory_alloc(
>       GST_GL_BASE_MEMORY_ALLOCATOR_CAST(allocator), 
> (GstGLAllocationParams*) params));
>
>     gst_gl_allocation_params_free((GstGLAllocationParams *)params);
>     gst_object_unref(allocator);
>
>     // Attach GstGLMemory object into buffer, timestamp the buffer and 
> push it downstream
>     GstBuffer* buffer = gst_buffer_new();
>     gst_buffer_append_memory(buffer, GST_MEMORY_CAST(glMemory));
>
>     // Put timestamps into buffer
>     GST_BUFFER_PTS (buffer) = timestamp;
>     GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int (1, 
> GST_SECOND, 2);
>
>     timestamp += GST_BUFFER_DURATION (buffer);
>     GstFlowReturn ret;
>     g_signal_emit_by_name(state->appsrc, "push-buffer", buffer, &ret);
>
>     if (ret != GST_FLOW_OK)
>     {
>         // Something wrong, stop pushing.
>         g_printerr("Something went wrong: Pushing buffer into appsrc 
> is stopped.\n");
>         return false;
>     }
>
>     return true;
> }
> /
>
> pushFrame produces the following error:
> gst_gl_context_thread_add: assertion 'context->priv->active_thread == 
> g_thread_self ()' failIed
>
> What am I doing wrong or how can push gpu texture to gstreamer?
>
> Thanks,
> Lusine
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210805/def2b203/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210805/def2b203/attachment.sig>