<div dir="ltr"><div dir="ltr">Hi Matthew,<br>Seems the second approach fits to my use case- I need to push textures which are created in opengles & egl( it means not in gstreamer context) to gstreamer.<div>I've changed my code to use
<code>gst_gl_display_create_context & </code>
<code><span>gst_gl_display_add_context <font face="arial, sans-serif">but still can't read textures in gstreamer, gstreamer prudeces the following errors after I push buffer to appsrc:</font></span></code></div><div><code>(testegl1:24602): GStreamer-CRITICAL **: 13:02:55.568: gst_debug_log_valist: assertion 'category != NULL' failed<br>0:00:14.039444188 24602 0x5611bc356d90 WARN glbasetexture gstglmemory.c:401:gst_gl_memory_read_pixels: Could not create framebuffer to read pixels for memory 0x7f8cf0017ac0<br>0:00:14.039480909 24602 0x5611bc356d90 WARN glbasememory gstglbasememory.c:585:gst_gl_base_memory_memcpy: could not read map source memory 0x7f8cf0017ac0<br>0:00:14.039505741 24602 0x5611bc356d90 WARN glmemory gstglmemorypbo.c:592:_gl_mem_copy: Could not copy GL Memory<br>0:00:14.039779268 24602 0x5611bc4f14f0 ERROR videometa gstvideometa.c:247:default_map: cannot map memory range 0-1<br>0:00:14.039846056 24602 0x5611bc4f14f0 ERROR default video-frame.c:168:gst_video_frame_map_id: failed to map video frame plane 0<br>0:00:14.039891314 24602 0x5611bc4f14f0 WARN videofilter gstvideofilter.c:297:gst_video_filter_transform:<videoconvert0> warning: invalid video buffer received<span><font face="arial, sans-serif"><br></font></span></code></div><div><code><span><font face="arial, sans-serif"><br></font></span></code></div><div><code><span><font face="arial, sans-serif"><br></font></span></code></div><div><code><span><font face="arial, sans-serif">This is how I implemented it, did I misunderstand something? </font></span></code></div><div><code><span><font face="arial, sans-serif"><br></font></span></code></div><div>I have rendering thread where I initialize opengles context and create wrapped and the new contexts.</div><div>//</div><div>// Description: Sets the display, OpenGL|ES context and screen stuff</div><div>// Created GstGLContext s - wrapped (gst_gl_context_new_wrapped) and new context(gst_gl_display_create_context)</div><div>//</div><div>static void<br>init_ogl (APP_STATE_T * state)<br></div><div>{</div><div>...</div> /* get an EGL display connection */<br> state->display = eglGetDisplay (EGL_DEFAULT_DISPLAY);<br> assert (state->display != EGL_NO_DISPLAY);<br> /* initialize the EGL display connection */<br> result = eglInitialize (state->display, NULL, NULL);<br><div> assert (EGL_FALSE != result); </div><div><br></div><div>/* create an EGL rendering context */<br> state->context =<br> eglCreateContext (state->display, config, EGL_NO_CONTEXT, context_attributes);<br> assert (state->context != EGL_NO_CONTEXT);<br></div><div>//</div><div>// Initialize GStreamer related resources.</div><div>//</div><div>state->gst_display = gst_gl_display_egl_new_with_egl_display (state->display);<br>state->gl_context =<br> gst_gl_context_new_wrapped (GST_GL_DISPLAY (state->gst_display),<br> (guintptr) state->context, GST_GL_PLATFORM_EGL, GST_GL_API_GLES2);<br><br>GError *error = NULL;<br>if ( !gst_gl_display_create_context(GST_GL_DISPLAY(state->gst_display), state->gl_context, &state->newContext, &error) )<br> g_print("Failed to create new context\n");</div><div><br>if ( !gst_gl_display_add_context(GST_GL_DISPLAY(state->gst_display), state->newContext))<br> g_print("Failed to add new context to display\n");</div><div><br></div><div>} // init_ogl end.</div><div><br></div><div><br></div><div>static void<br>sync_bus_call (GstBus * bus, GstMessage * msg, gpointer * data)<br>{<br><br> APP_STATE_T *state = (APP_STATE_T *)data;<br> switch (GST_MESSAGE_TYPE (msg))<br> {<br> case GST_MESSAGE_NEED_CONTEXT:<br> {<br> const gchar *context_type;<br> gst_message_parse_context_type (msg, &context_type);<br><br> GstContext *context = NULL;<br> if (g_strcmp0 (context_type, GST_GL_DISPLAY_CONTEXT_TYPE) == 0)<br> {<br> GstGLDisplay * gl_display = GST_GL_DISPLAY(state->gst_display);<br> context = gst_context_new (GST_GL_DISPLAY_CONTEXT_TYPE, TRUE);<br> gst_context_set_gl_display(context, gl_display);<br> gst_element_set_context (GST_ELEMENT(msg->src), context);<br> }<br> else if (g_strcmp0 (context_type, "gst.gl.app_context") == 0)<br> {<br> GstContext *context = gst_context_new("gst.gl.app_context", TRUE);<br> GstStructure *s = gst_context_writable_structure (context);<br> gst_structure_set (s, "context", GST_TYPE_GL_CONTEXT, state->gl_context, NULL);<br> gst_element_set_context(GST_ELEMENT(msg->src), context); <br> }<br> break;<br> }<br> default:<br> break;<br> }<br>} // sync_bus_call end<br></div><div><br></div><div><br></div><div><div>I use need-data callback to create a buffer from texture_id and and push it in the appsrc:</div><div><i>g_signal_connect (state->appsrc, "need-data", G_CALLBACK (</i> pushFrame <i>), state);</i></div></div><div><i><br></i></div><div>static bool pushFrame(..., APP_STATE_T * state)<br>{<br> // Wrap the texture into GstGLMemory<br> GstVideoInfo vinfo;<br> gst_video_info_set_format(&vinfo, GST_VIDEO_FORMAT_RGBA, state->screen_width, state->screen_height);<br> // Use
state->newContext for allocator.<br> GstAllocator* allocator = GST_ALLOCATOR(gst_gl_memory_allocator_get_default(state->newContext));<br> GstGLVideoAllocationParams* params = gst_gl_video_allocation_params_new_wrapped_texture(<br> state->newContext, NULL, &vinfo, 0, NULL, GST_GL_TEXTURE_TARGET_2D, GST_GL_RGBA, state->tex,<br> NULL, 0);<br><br> GstGLMemory* glMemory = GST_GL_MEMORY_CAST(gst_gl_base_memory_alloc(<br> GST_GL_BASE_MEMORY_ALLOCATOR_CAST(allocator), (GstGLAllocationParams*) params));<br><br> gst_gl_allocation_params_free((GstGLAllocationParams *)params);<br> gst_object_unref(allocator);<br><br> // Attach GstGLMemory object into buffer, timestamp the buffer and push it downstream<br> GstBuffer* buffer = gst_buffer_new();<br> gst_buffer_append_memory(buffer, GST_MEMORY_CAST(glMemory));<br><br> GstVideoMeta * vmeta = gst_buffer_add_video_meta(buffer, GST_VIDEO_FRAME_FLAG_NONE, GST_VIDEO_FORMAT_RGBA, state->screen_width, state->screen_height);<br><br> // Put timestamps into buffer<br> GST_BUFFER_PTS (buffer) = timestamp;<br> GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int (1, GST_SECOND, 2);<br> timestamp += GST_BUFFER_DURATION (buffer);<br><br> GstFlowReturn ret;<br> g_signal_emit_by_name(state->appsrc, "push-buffer", buffer, &ret);<br><br> if (ret != GST_FLOW_OK)<br> {<br> // Something wrong, stop pushing.<br> g_printerr("Something went wrong: Pushing buffer into appsrc is stopped.\n");<br> return false;<br> }<br><br> return true;<br>} // pushFrame end<br><br></div><div>Regards,</div><div>Lusine</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Aug 6, 2021 at 8:59 AM Matthew Waters <<a href="mailto:ystreet00@gmail.com" target="_blank">ystreet00@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
Hi,<br>
<br>
<div>On 6/8/21 1:12 am, Lusine Hayrapetyan
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Dear Matt,<br>
Thank you very much for your response. It helped me to
understand that using 'wrapped' OpenGL context is a wrong
direction to go :)
<div><br>
<i>WRT Suggestion 1:</i>
<div>Do I understand correctly that I need to get local
context from gstreamer and pass it to opengl rendering
thread? (I have a rendering thread where I set OpenGL|ES
context and screen stuff).</div>
<div>GstGLContext* mContext = nullptr;<br>
</div>
<div>g_object_get(state->gldownload, "context",
&mContext, NULL);<br>
</div>
<div>guintptr handle;<br>
handle = gst_gl_context_get_gl_context(mContext); // is this
correct?<br>
state->context = (EGLContext)handle; // state->context
is EGLContext type;<br>
</div>
<div>And then use state->context in OpenGL|ES?</div>
</div>
<div>Do I need to get and pass window and display from gstreamer
to my rendering thread as well?</div>
<div>Although my use scenario is different from this one - I
need to pass context from OpenGL to Gstreamer.</div>
</div>
</blockquote>
<br>
You only need to retrieve or create a non-wrapped GstGLContext and
use that for creating your textures that you are pushing into
GStreamer. You don't need to use GStreamer's provided OpenGL
context for anything else. Everything else in your sample remains
the same. You may need to add a GstGLSyncMeta on your buffers you
are pushing into GStreamer to provide the necessary synchronisation
guarantees between the shared OpenGL contexts (application and
GStreamer). On some platforms the window handle type and format may
be important however in general on linux (X11/wayland) it doesn't
really matter.<br>
<br>
You must not attempt to use GStreamer's OpenGL context as-is (using
e.g. eglMakeCurrent() or anything of the like) from outside the
GStreamer OpenGL context thread (as provided by the
gst_gl_context_thread_add() API).<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div><i>WRT Suggestion 2:</i><br>
</div>
<div>gst_gl_display_create_context accepts <i>other_context </i>argument,
should the <i>other_context </i>be the 'wrapped' context?</div>
</div>
</blockquote>
<br>
Yes. other_context is the GstGLContext that will be shared with the
newly created GstGLContext.<br>
<br>
Side note, GStreamer cannot use any application-provided OpenGL
context as-is due to the overhead of dealing with all the OpenGL
state that may be changed behind GStreamer's back. This is why the
OpenGL context sharing dance is required.<br>
<br>
Cheers<br>
-Matt<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>Best Regards,</div>
<div>Lusine</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Aug 5, 2021 at 12:39
PM Matthew Waters <<a href="mailto:ystreet00@gmail.com" target="_blank">ystreet00@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> So, I think you almost have the correct sequence.<br>
<br>
Response inline.<br>
<br>
<div>On 5/8/21 1:04 am, Lusine Hayrapetyan via
gstreamer-devel wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Folks,
<div>I'm struggling with the following issue and can't
understand what I'm doing wrong.</div>
<div>I need to pass opengl texture to the gstreamer
pipeline.</div>
<div>I have a rendering thread where I create opengl
texture, the following objects created in this thread:</div>
<div><i>EGLDisplay display;</i></div>
<div><i>EGLContext context;</i><br>
</div>
<div><i><br>
</i></div>
<div>I create gstreamer pipeline in the main thread and
as described in the following article sharing an X11
display and GstGLContext with the bus callback.</div>
<div><a href="http://ystreet00.blogspot.com/2015/09/gstreamer-16-and-opengl-contexts.html" target="_blank">http://ystreet00.blogspot.com/2015/09/gstreamer-16-and-opengl-contexts.html</a><br>
</div>
<div><br>
</div>
<div>GstGLDisplayEGL and GstGLContext are created in
this way:</div>
<div><i>GstGLDisplayEGL* gst_display =
gst_gl_display_egl_new_with_egl_display (display);<br>
</i></div>
<div><i>GstGLContext *gl_context =<br>
gst_gl_context_new_wrapped (GST_GL_DISPLAY
(gst_display),<br>
(guintptr) context, GST_GL_PLATFORM_EGL,
GST_GL_API_GLES2);</i><br>
</div>
<div><br>
</div>
<div>The first element of my pipeline is appsrc:</div>
<div><i>appsrc stream-type=0 emit-signals=1 format=3
caps=video/x-raw(memory:GLMemory), width=300,
height=300, framerate=(fraction)20/1,
format=(string)RGBA ! gldownload ! ...</i><br>
</div>
<div><br>
</div>
<div>I use need-data callback to create a buffer from
texture_id and and push it in the appsrc:</div>
<div><i>g_signal_connect (state->appsrc, "need-data",
G_CALLBACK (</i> pushFrame <i>), state);</i></div>
<div><i><br>
</i></div>
<div><i>bool pushFrame()</i><br>
</div>
<div><i>{<br>
// Wrap the texture into GstGLMemory<br>
GstVideoInfo vinfo;<br>
gst_video_info_set_format(&vinfo,
GST_VIDEO_FORMAT_RGBA, 300, 300);<br>
<br>
GstAllocator* allocator =
GST_ALLOCATOR(gst_gl_memory_allocator_get_default(gl_context));<br>
<br>
GstGLVideoAllocationParams* params =
gst_gl_video_allocation_params_new_wrapped_texture(<br>
state->gl_context, NULL, &vinfo, 0,
NULL, GST_GL_TEXTURE_TARGET_2D, GST_GL_RGBA, </i>
texture_id <i>,<br>
NULL, 0);<br>
</i></div>
</div>
</blockquote>
<br>
The use of state->gl_context is probably your OpenGL
context that has been wrapped from the application. This
'wrapped' OpenGL context has some limitations, one being
that GStreamer cannot actually do a complete
gst_gl_context_thread_add where the request is marshalled to
an OpenGL-specific thread. This is what that critical is
complaining about effectively.<br>
<br>
To do this properly, you would need to do one of two things:<br>
1. Retrieve the OpenGL context from the downstream
gldownload element using either the 'context' property or
using an appropriate GST_CONTEXT QUERY or the helper
gst_gl_query_local_gl_context().<br>
2. Create your own GStreamer OpenGL context and add it to
the GstGLDisplay using something like: <a href="https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/gst-libs/gst/gl/gstglbasefilter.c#L550-565" target="_blank">https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/gst-libs/gst/gl/gstglbasefilter.c#L550-565</a>.<br>
<br>
Cheers<br>
-Matt<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div><i><br>
// The following line produces an error!!!<br>
GstGLMemory* glMemory =
GST_GL_MEMORY_CAST(gst_gl_base_memory_alloc(<br>
GST_GL_BASE_MEMORY_ALLOCATOR_CAST(allocator),
(GstGLAllocationParams*) params));<br>
<br>
gst_gl_allocation_params_free((GstGLAllocationParams
*)params);<br>
gst_object_unref(allocator);<br>
<br>
// Attach GstGLMemory object into buffer,
timestamp the buffer and push it downstream<br>
GstBuffer* buffer = gst_buffer_new();<br>
gst_buffer_append_memory(buffer,
GST_MEMORY_CAST(glMemory));<br>
<br>
// Put timestamps into buffer<br>
GST_BUFFER_PTS (buffer) = timestamp;<br>
GST_BUFFER_DURATION (buffer) =
gst_util_uint64_scale_int (1, GST_SECOND, 2);<br>
<br>
timestamp += GST_BUFFER_DURATION (buffer);<br>
GstFlowReturn ret;<br>
g_signal_emit_by_name(state->appsrc,
"push-buffer", buffer, &ret);<br>
<br>
if (ret != GST_FLOW_OK)<br>
{<br>
// Something wrong, stop pushing.<br>
g_printerr("Something went wrong: Pushing
buffer into appsrc is stopped.\n");<br>
return false;<br>
}<br>
<br>
return true;<br>
}<br>
</i></div>
<div><br>
</div>
<div>pushFrame produces the following error:</div>
<div><font color="#ff0000">gst_gl_context_thread_add:
assertion 'context->priv->active_thread ==
g_thread_self ()' failIed<br>
</font></div>
<div><font color="#ff0000"><br>
</font></div>
<div><font color="#000000">What am I doing wrong or how
can push gpu texture to gstreamer?</font></div>
<div><font color="#000000"><br>
</font></div>
<div><font color="#000000">Thanks,</font></div>
<div><font color="#000000">Lusine</font></div>
<div><font color="#000000"><br>
</font></div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
</blockquote>
<br>
</div>
</blockquote></div></div>