push opengl texture to gstreamer pipeline

[Lusine Hayrapetyan]

Dear Matt,
Thank you very much for your response. It helped me to understand that
using  'wrapped' OpenGL context is a wrong direction to go :)

*WRT Suggestion 1:*
Do I understand correctly that I need to get local context from gstreamer
and pass it to opengl rendering thread? (I have a rendering thread where I
set OpenGL|ES context and screen stuff).
GstGLContext* mContext = nullptr;
g_object_get(state->gldownload, "context", &mContext, NULL);
guintptr handle;
handle = gst_gl_context_get_gl_context(mContext); // is this correct?
state->context = (EGLContext)handle; // state->context is EGLContext type;
And then use state->context in OpenGL|ES?
Do I need to get and pass window and display from gstreamer to my rendering
thread as well?
Although my use scenario is different from this one - I need to pass
context from OpenGL to Gstreamer.

*WRT Suggestion 2:*
gst_gl_display_create_context accepts *other_context *argument, should the
*other_context *be the 'wrapped' context?

Best Regards,
Lusine

On Thu, Aug 5, 2021 at 12:39 PM Matthew Waters <ystreet00 at gmail.com> wrote:

> So, I think you almost have the correct sequence.
>
> Response inline.
>
> On 5/8/21 1:04 am, Lusine Hayrapetyan via gstreamer-devel wrote:
>
> Hi Folks,
> I'm struggling with the following issue and can't understand what I'm
> doing wrong.
> I need to pass opengl texture to the gstreamer pipeline.
> I have a rendering thread where I create opengl texture, the following
> objects created in this thread:
> *EGLDisplay display;*
> *EGLContext context;*
>
> I create gstreamer pipeline in the main thread and as described in the
> following article sharing an X11 display and GstGLContext with the bus
> callback.
> http://ystreet00.blogspot.com/2015/09/gstreamer-16-and-opengl-contexts.html
>
> GstGLDisplayEGL and GstGLContext are created in this way:
>
> *GstGLDisplayEGL* gst_display = gst_gl_display_egl_new_with_egl_display
> (display); *
>
>
> *GstGLContext *gl_context =       gst_gl_context_new_wrapped
> (GST_GL_DISPLAY (gst_display),       (guintptr) context,
> GST_GL_PLATFORM_EGL, GST_GL_API_GLES2);*
>
> The first element of my pipeline is appsrc:
> *appsrc stream-type=0 emit-signals=1 format=3
> caps=video/x-raw(memory:GLMemory),  width=300, height=300,
> framerate=(fraction)20/1, format=(string)RGBA ! gldownload ! ...*
>
> I use need-data callback to create a buffer from texture_id and and push
> it in the appsrc:
> *g_signal_connect (state->appsrc, "need-data", G_CALLBACK (* pushFrame *),
> state);*
>
> *bool pushFrame()*
>
>
>
>
>
>
>
>
> *{      // Wrap the texture into GstGLMemory     GstVideoInfo vinfo;
> gst_video_info_set_format(&vinfo, GST_VIDEO_FORMAT_RGBA, 300, 300);
> GstAllocator* allocator =
> GST_ALLOCATOR(gst_gl_memory_allocator_get_default(gl_context));
> GstGLVideoAllocationParams* params =
> gst_gl_video_allocation_params_new_wrapped_texture(
>  state->gl_context, NULL, &vinfo, 0, NULL, GST_GL_TEXTURE_TARGET_2D,
> GST_GL_RGBA, * texture_id
>
> *,       NULL, 0); *
>
>
> The use of state->gl_context is probably your OpenGL context that has been
> wrapped from the application.  This 'wrapped' OpenGL context has some
> limitations, one being that GStreamer cannot actually do a complete
> gst_gl_context_thread_add where the request is marshalled to an
> OpenGL-specific thread.  This is what that critical is complaining about
> effectively.
>
> To do this properly, you would need to do one of two things:
> 1. Retrieve the OpenGL context from the downstream gldownload element
> using either the 'context' property or using an appropriate GST_CONTEXT
> QUERY or the helper gst_gl_query_local_gl_context().
> 2. Create your own GStreamer OpenGL context and add it to the GstGLDisplay
> using something like:
> https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/gst-libs/gst/gl/gstglbasefilter.c#L550-565
> .
>
> Cheers
> -Matt
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *     // The following line produces an error!!!     GstGLMemory* glMemory
> = GST_GL_MEMORY_CAST(gst_gl_base_memory_alloc(
> GST_GL_BASE_MEMORY_ALLOCATOR_CAST(allocator), (GstGLAllocationParams*)
> params));     gst_gl_allocation_params_free((GstGLAllocationParams
> *)params);     gst_object_unref(allocator);     // Attach GstGLMemory
> object into buffer, timestamp the buffer and push it downstream
> GstBuffer* buffer = gst_buffer_new();     gst_buffer_append_memory(buffer,
> GST_MEMORY_CAST(glMemory));     // Put timestamps into buffer
> GST_BUFFER_PTS (buffer) = timestamp;     GST_BUFFER_DURATION (buffer) =
> gst_util_uint64_scale_int (1, GST_SECOND, 2);     timestamp +=
> GST_BUFFER_DURATION (buffer);     GstFlowReturn ret;
> g_signal_emit_by_name(state->appsrc, "push-buffer", buffer, &ret);     if
> (ret != GST_FLOW_OK)     {         // Something wrong, stop pushing.
>   g_printerr("Something went wrong: Pushing buffer into appsrc is
> stopped.\n");         return false;     }     return true; } *
>
> pushFrame produces the following error:
> gst_gl_context_thread_add: assertion 'context->priv->active_thread ==
> g_thread_self ()' failIed
>
> What am I doing wrong or how can push gpu texture to gstreamer?
>
> Thanks,
> Lusine
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210805/f5a81ded/attachment.htm>