<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> </head> <body> Hi,<br> <br> I think I'm at the end if what I can debug over email. In theory everything should work and GStreamer has examples and elements that do this exact process of sharing the OpenGL context between GStreamer and the application/other GUI toolkits.<br> <br> Please look through the debug logs and any existing applications/elements (qmlglsink, gtkglsink) that perform this transformation from the application provided OpenGL context (e.g. from Qt or Gtk).<br> <br> <div class="moz-cite-prefix">On 18/8/21 12:13 am, Lusine Hayrapetyan wrote:<br> </div> <blockquote type="cite" cite="mid:CAO4TkMiu7UB57TMnV9TeQA+b70TO_ct=O2tqqUyVazcZ9frFmA@mail.gmail.com"> <meta http-equiv="content-type" content="text/html; charset=UTF-8"> <div dir="ltr">WRT <i>" retrieve the GStreamer created OpenGL context from one of the downstream elements" </i>unfortunately it's not our use case. <br> </div> </blockquote> <br> If it's not your case, you can also perform the necessary GST_CONTEXT QUERY to retrieve the downstream OpenGL context from GStreamer.<br> <br> <blockquote type="cite" cite="mid:CAO4TkMiu7UB57TMnV9TeQA+b70TO_ct=O2tqqUyVazcZ9frFmA@mail.gmail.com"> <div dir="ltr"> <div> <div>WRT " <i>If you are not also providing your wrapped OpenGL in response to the 'need-context' message" </i>I'm providing it, this is how I handle "need-context":</div> <div>static void sync_bus_call (GstBus * bus, GstMessage * msg, gpointer * data)<br> {<br> APP_STATE_T *state = (APP_STATE_T *)data;<br> switch (GST_MESSAGE_TYPE (msg))<br> {<br> case GST_MESSAGE_NEED_CONTEXT:<br> {<br> g_print("-------------------------sync_bus_call got GST_MESSAGE_NEED_CONTEXT--------------------\n");<br> <br> </div> <div> const gchar *context_type;<br> gst_message_parse_context_type (msg, &context_type);<br> <br> GstContext *context = NULL;<br> if (g_strcmp0 (context_type, GST_GL_DISPLAY_CONTEXT_TYPE) == 0)<br> {<br> g_print("-------------------------got GST_GL_DISPLAY_CONTEXT_TYPE--------------------\n");<br> context = gst_context_new (GST_GL_DISPLAY_CONTEXT_TYPE, TRUE);</div> <div><br> </div> <div> // state->x_gst_display was created like this: state->x_gst_display = gst_gl_display_x11_new_with_display(state->xdisplay);<br> gst_context_set_gl_display(context, GST_GL_DISPLAY(state->x_gst_display));<br> <br> g_print("msg->src element is %s\n\n", GST_ELEMENT_NAME(msg->src)); // This prints gldownload.<br> gst_element_set_context (GST_ELEMENT(msg->src), context);<br> }<br> else if (g_strcmp0 (context_type, "gst.gl.app_context") == 0)<br> {<br> g_print("-------------------------got gst.gl.app_context--------------------\n");<br> <br> GstContext *context = gst_context_new("gst.gl.app_context", TRUE);<br> GstStructure *s = gst_context_writable_structure (context);</div> <div><br> </div> <div> // state->gl_context is my wrapped context which was created like this:</div> <div> // state->gl_context = gst_gl_context_new_wrapped (GST_GL_DISPLAY (state->x_gst_display),<br> // (guintptr) state->context, GST_GL_PLATFORM_EGL, GST_GL_API_GLES2);<br> gst_structure_set (s, "context", GST_TYPE_GL_CONTEXT, state->gl_context, NULL);<br> <br> g_print("msg->src element is %s\n", GST_ELEMENT_NAME(msg->src)); // This prints gldownload. <br> gst_element_set_context(GST_ELEMENT(msg->src), context);<br> }<br> break;<br> }<br> </div> </div> </div> </blockquote> <br> This looks correct from a cursory glance.<br> <br> <blockquote type="cite" cite="mid:CAO4TkMiu7UB57TMnV9TeQA+b70TO_ct=O2tqqUyVazcZ9frFmA@mail.gmail.com"> <div dir="ltr"> <div> <div>WRT: <i>"if you are creating another unwrapped OpenGL context with gst_gl_display_create_context() then you have notify GStreamer about this somehow"</i></div> <div>I use the unwrapped gl context with <i>gst_gl_display_add_context </i>and when I'm creating GstGLMemory, is there other places I need to use it?<i> <br> </i></div> </div> </div> </blockquote> <br> No. That should be sufficient.<br> <br> Cheers<br> -Matt<br> <br> <blockquote type="cite" cite="mid:CAO4TkMiu7UB57TMnV9TeQA+b70TO_ct=O2tqqUyVazcZ9frFmA@mail.gmail.com"> <div dir="ltr"> <div> <div>Best Regards,</div> <div>Lusine</div> </div> <div><br> </div> </div> <br> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">On Tue, Aug 17, 2021 at 4:52 PM Matthew Waters <<a href="mailto:ystreet00@gmail.com" moz-do-not-send="true">ystreet00@gmail.com</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div> Ah,<br> <br> Ok. Now, if you are creating another unwrapped OpenGL context with gst_gl_display_create_context() then you have notify GStreamer about this somehow, or GStreamer will also create it's own. If you are not also providing your wrapped OpenGL in response to the 'need-context' message, then GStreamer OpenGL is not linked with your application OpenGL context.<br> <br> You should probably just retrieve the GStreamer created OpenGL context from one of the downstream elements (the 'context' property on most OpenGL elements) and use that.<br> <br> Cheers<br> -Matt<br> <br> <div>On 17/8/21 1:53 am, Lusine Hayrapetyan wrote:<br> </div> <blockquote type="cite"> <div dir="ltr">Got it! Thanks!<br> Used <i>gst_gl_display_x11_new_with_display </i>instead of <i>gst_gl_display_egl_new_with_egl_display </i>and the warning about " <i>Wrong display type" </i>is fixed. I don't see any other warn/issue in the gl log but my initial issue still exists 😿 - I create GstGLMemory from texture_id and push the GstBuffer to appsrc and see this errors: <div><br> <div>0:00:14.023472773 306156 0x55a56659d6a0 LOG glbasememory gstglbasememory.c:269:_map_data_gl: mapping mem 0x7f105c227d00 flags 20002<br> 0:00:14.023553671 306156 0x55a56659d6a0 DEBUG gldebug gstgldebug.c:319:_gst_gl_debug_callback:<glcontextegl0> medium: GL other from API id:1, FBO incomplete: color attachment incomplete [0]<br> <br> (testegl1:306156): GStreamer-CRITICAL **: 15:39:40.709: gst_debug_log_valist: assertion 'category != NULL' failed<br> 0:00:14.023741217 306156 0x55a56659d6a0 WARN glbasetexture gstglmemory.c:404:gst_gl_memory_read_pixels: Could not create framebuffer to read pixels for memory 0x7f10d08992b0<br> 0:00:14.023776145 306156 0x55a56659d6a0 LOG glbasememory gstglbasememory.c:348:_unmap_data_gl: unmapping mem 0x7f105c227d00 flags 20002<br> 0:00:14.023800095 306156 0x55a56659d6a0 LOG glbuffer gstglbuffer.c:305:_gl_buffer_unmap: unmapping 0x7f105c227d00 id 1 size 360000<br> 0:00:14.024028993 306156 0x55a56659d6a0 LOG glbasememory gstglbasememory.c:269:_map_data_gl: mapping mem 0x7f10d08992b0 flags 10001<br> 0:00:14.024074629 306156 0x55a56659d6a0 LOG glbasememory gstglbasememory.c:227:gst_gl_base_memory_alloc_data: 0x7f10d08992b0 attempting allocation of data pointer of size 360007<br> 0:00:14.024139538 306156 0x55a56659d6a0 DEBUG glbasememory gstglbasememory.c:236:gst_gl_base_memory_alloc_data: 0x7f10d08992b0 allocated data pointer alloc 0x7f10c05fa010, data 0x7f10c05fa010<br> 0:00:14.024169028 306156 0x55a56659d6a0 LOG glbasememory gstglbasememory.c:269:_map_data_gl: mapping mem 0x7f105c227d00 flags 20002<br> 0:00:14.024195194 306156 0x55a56659d6a0 LOG glbasememory gstglbasememory.c:269:_map_data_gl: mapping mem 0x7f105c227d00 flags 20002<br> 0:00:14.024217673 306156 0x55a56659d6a0 LOG glbasememory gstglbasememory.c:278:_map_data_gl: multiple map no 2 flags 20002 all flags 20002<br> 0:00:14.024263516 306156 0x55a56659d6a0 DEBUG gldebug gstgldebug.c:319:_gst_gl_debug_callback:<glcontextegl0> medium: GL other from API id:1, FBO incomplete: color attachment incomplete [0]<br> </div> </div> </div> <br> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">On Mon, Aug 16, 2021 at 6:55 PM Matthew Waters <<a href="mailto:ystreet00@gmail.com" target="_blank" moz-do-not-send="true">ystreet00@gmail.com</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div> If you're running on linux, there is a native display handle behind the EGLDIsplay. That native display handle would be an X11 Display, or a wayland wl_display (or maybe a mir display). OpenGL resources cannot be shared unless this 'native' display type is the same between all OpenGL uses. Passing an EGLDisplay is not sufficient and will not work.<br> <br> You need to pass the state->xdisplay using the appropriate platform-specific gst_gl_display_x11_new_with_display(). You cannot use the EGLDisplay version here.<br> <br> Cheers<br> -Matt<br> <br> <div>On 17/8/21 12:40 am, Lusine Hayrapetyan wrote:<br> </div> <blockquote type="cite"> <div dir="ltr">Hi Mat, <div><br> </div> <div>My display is <b>EGLDisplay </b>type:<br> <div> /* get an EGL display connection */<br> </div> <div> <div> EGLNativeDisplayType nativeDisplay = state->xdisplay;<br> </div> <div> <b>state->display</b> = eglGetDisplay (nativeDisplay);<br> </div> assert (state->display != EGL_NO_DISPLAY);</div> <div><br> </div> <div>/* The I create an EGL rendering context with EGLDisplay */<br> state->context =<br> eglCreateContext (state->display, config, EGL_NO_CONTEXT, context_attributes);<br> </div> <div><br> </div> </div> <div>/* Then I create GstGLDisplayEGL from EGLDisplay */</div> <div>state->gst_display = gst_gl_display_egl_new_with_egl_display (state->display);<br> </div> <div><br> </div> <div>/* And create my wrapped context with GstGLDisplayEGL */</div> <div> state->gl_context =<br> gst_gl_context_new_wrapped (GST_GL_DISPLAY (state->gst_display),<br> (guintptr) state->context, GST_GL_PLATFORM_EGL, GST_GL_API_GLES2);<br> </div> <div><br> </div> <div>/* Now I want to create a shared context from my wrapped context and again I pass GstGLDisplayEGL to it */</div> <div>gst_gl_display_create_context(GST_GL_DISPLAY(state->gst_display), state->gl_context, &state->newContext, &error) )<br> </div> <div><br> </div> <div>And this function complains that it expects an x11 display instead of egl display. But I don't understand how I can pass x11 display because my context is created with egl display.</div> <div>I'm not experienced in opengl es & egl, do I miss something?</div> <div><br> </div> <div>Regards,</div> <div>Lusine</div> </div> <br> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">On Fri, Aug 13, 2021 at 1:48 PM Matthew Waters <<a href="mailto:ystreet00@gmail.com" target="_blank" moz-do-not-send="true">ystreet00@gmail.com</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div> <br> <br> <div>On 13/8/21 5:59 pm, Lusine Hayrapetyan wrote:<br> </div> <blockquote type="cite"> <div dir="ltr">In gstglconfig.h I see the following: <div><i>#define GST_GL_HAVE_WINDOW_X11 1<br> </i></div> <div><br> </div> <div>I've built the latest master on Ubuntu 20 and still see the same message about x11 window and there is a new FIXME message:<br> 0:00:03.382786440 252231 0x55e6075596a0 INFO glwindow gstglwindow.c:295:gst_gl_window_new: creating a window, user choice:x11<br> 0:00:03.382847275 252231 0x55e6075596a0 INFO glwindow gstglwindow_x11.c:136:gst_gl_window_x11_new: Wrong display type 32 for this window type 1<br> </div> </div> </blockquote> <br> This is the problematic line. The display type (EGL) as returned by gst_gl_display_get_type() does not match the window impl's expectations (X11). If you are running under X11 and your application is using X11 to connect to the display server, you need to pass an X11 GstGLDisplay. Passing an EGLDisplay is not sufficient and will not work. An appropriate EGLDIsplay will be created internally.<br> <br> Cheers<br> -Matt<br> <br> <blockquote type="cite"> <div dir="ltr"> <div>0:00:03.382877455 252231 0x55e6075596a0 WARN <font color="#ff0000"> glwindow gstglwindow.c:340:gst_gl_window_new: Could not create window. user specified x11, creating dummy window</font><br> 0:00:03.382998854 252231 0x55e6075596a0 DEBUG gldisplay gstgldisplay.c:694:gst_gl_display_create_window:<gldisplayegl0> Adding window <gldummywindow0> (0x55e607797c50) to internal list<br> 0:00:03.383024753 252231 0x55e6075596a0 DEBUG glcontext gstglcontext.c:963:gst_gl_context_set_window:<glcontextegl0> window:<gldummywindow0><br> 0:00:03.383269595 252231 0x55e6077b5640 DEBUG glcontext gstglcontext.c:1223:gst_gl_context_create_thread:<glcontextegl0> Creating thread<br> 0:00:03.383377039 252231 0x55e6077b5640 <font color="#ff0000">FIXME glcontext gstglcontext.c:2039:gst_gl_wrapped_context_get_config:<glwrappedcontext0> wrapped context could not retrieve config. The application may be missing a call to gst_gl_context_fill_info() or the specific platform implemention is not implemented for retrieving the config from a wrapped OpenGL context.</font><br> 0:00:03.383437907 252231 0x55e6077b5640 INFO glcontext gstglcontext.c:1315:gst_gl_context_create_thread:<glcontextegl0> Attempting to create opengl context. user chosen api(s) (gles2), compiled api support (opengl opengl3 gles2) display api (gles2)<br> 0:00:03.383463315 252231 0x55e6077b5640 DEBUG glcontext gstglcontext_egl.c:863:gst_gl_context_egl_create_context:<glcontextegl0> Creating EGL context<br> </div> <div><br> </div> <div>Regards,</div> <div>Lusine</div> </div> <br> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">On Thu, Aug 12, 2021 at 11:18 AM Matthew Waters <<a href="mailto:ystreet00@gmail.com" target="_blank" moz-do-not-send="true">ystreet00@gmail.com</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div> Yes, that is a critical error for X11. X11 would require access to an X11 Display which must be provided by your application in order share OpenGL resources. You may also not have X11 support built and you would have to check your build configuration for that. gstglconfig.h would contain information on what libgstgl-1.0 was built with.<br> <br> It looks like it's picking a raw EGLDisplay instead. This was potentially an issue in previous versions of GStreamer that has been fixed in master by <a href="https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1154" target="_blank" moz-do-not-send="true">https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1154</a> and <a href="https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1169" target="_blank" moz-do-not-send="true">https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1169</a>.<br> <br> Cheers<br> -Matt<br> <br> <div>On 12/8/21 1:50 am, Lusine Hayrapetyan wrote:<br> </div> <blockquote type="cite"> <div dir="ltr">For limiting to a particular OpenGL variant I'm using the following environment variables, is it the same? <div><i> export GST_GL_WINDOW=x11<br> export GST_GL_PLATFORM=egl<br> export GST_GL_API=gles2</i><br> </div> <div><i><br> </i></div> <div>I've attached GST_DEBUG=gl*:7 output (gl_output.txt).</div> <div>I see the following messages in the output:<br> <i>0:00:03.857084617 24438 0x55d59121d400 INFO glwindow gstglwindow.c:278:gst_gl_window_new: creating a window, user choice:x11<br> 0:00:03.857117499 24438 0x55d59121d400 INFO glwindow gstglwindow_x11.c:137:gst_gl_window_x11_new: Wrong display type 32 for this window type 1<br> 0:00:03.857144332 24438 0x55d59121d400 WARN glwindow gstglwindow.c:324:gst_gl_window_new: Could not create window. user specified x11, creating dummy window<br> 0:00:03.857277761 24438 0x55d59121d400 DEBUG gldisplay gstgldisplay.c:579:gst_gl_display_create_window:<gldisplayegl0> Adding window <gldummywindow0> (0x55d5913809d0) to internal list<br> 0:00:03.857311159 24438 0x55d59121d400 DEBUG glcontext gstglcontext.c:948:gst_gl_context_set_window:<glcontextegl0> window:<gldummywindow0></i><br> </div> <div><i><br> </i></div> <div>Is this a critical issue? Can this cause a texture reading issue?</div> </div> <br> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">On Tue, Aug 10, 2021 at 10:21 AM Matthew Waters <<a href="mailto:ystreet00@gmail.com" target="_blank" moz-do-not-send="true">ystreet00@gmail.com</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div> Ah, if you want to limit to a particular OpenGL variant, you need to also call gst_gl_display_filter_gl_api() with that variant.<br> <br> GST_DEBUG=gl*:7 will output a whole bunch of information about what is generated and tried in all of this respect.<br> <br> Cheers<br> -Matt<br> <br> <div>On 9/8/21 12:58 am, Lusine Hayrapetyan wrote:<br> </div> <blockquote type="cite"> <div dir="ltr"> <div dir="ltr">Hi Matthew,<br> Seems the second approach fits to my use case- I need to push textures which are created in opengles & egl( it means not in gstreamer context) to gstreamer. <div>I've changed my code to use <code>gst_gl_display_create_context & </code> <code><span>gst_gl_display_add_context <font face="arial, sans-serif">but still can't read textures in gstreamer, gstreamer prudeces the following errors after I push buffer to appsrc:</font></span></code></div> <div><code>(testegl1:24602): GStreamer-CRITICAL **: 13:02:55.568: gst_debug_log_valist: assertion 'category != NULL' failed<br> 0:00:14.039444188 24602 0x5611bc356d90 WARN glbasetexture gstglmemory.c:401:gst_gl_memory_read_pixels: Could not create framebuffer to read pixels for memory 0x7f8cf0017ac0<br> 0:00:14.039480909 24602 0x5611bc356d90 WARN glbasememory gstglbasememory.c:585:gst_gl_base_memory_memcpy: could not read map source memory 0x7f8cf0017ac0<br> 0:00:14.039505741 24602 0x5611bc356d90 WARN glmemory gstglmemorypbo.c:592:_gl_mem_copy: Could not copy GL Memory<br> 0:00:14.039779268 24602 0x5611bc4f14f0 ERROR videometa gstvideometa.c:247:default_map: cannot map memory range 0-1<br> 0:00:14.039846056 24602 0x5611bc4f14f0 ERROR default video-frame.c:168:gst_video_frame_map_id: failed to map video frame plane 0<br> 0:00:14.039891314 24602 0x5611bc4f14f0 WARN videofilter gstvideofilter.c:297:gst_video_filter_transform:<videoconvert0> warning: invalid video buffer received<span><font face="arial, sans-serif"><br> </font></span></code></div> <div><code><span><font face="arial, sans-serif"><br> </font></span></code></div> <div><code><span><font face="arial, sans-serif"><br> </font></span></code></div> <div><code><span><font face="arial, sans-serif">This is how I implemented it, did I misunderstand something? </font></span></code></div> <div><code><span><font face="arial, sans-serif"><br> </font></span></code></div> <div>I have rendering thread where I initialize opengles context and create wrapped and the new contexts.</div> <div>//</div> <div>// Description: Sets the display, OpenGL|ES context and screen stuff</div> <div>// Created GstGLContext s - wrapped (gst_gl_context_new_wrapped) and new context(gst_gl_display_create_context)</div> <div>//</div> <div>static void<br> init_ogl (APP_STATE_T * state)<br> </div> <div>{</div> <div>...</div> /* get an EGL display connection */<br> state->display = eglGetDisplay (EGL_DEFAULT_DISPLAY);<br> assert (state->display != EGL_NO_DISPLAY);<br> /* initialize the EGL display connection */<br> result = eglInitialize (state->display, NULL, NULL);<br> <div> assert (EGL_FALSE != result); </div> <div><br> </div> <div>/* create an EGL rendering context */<br> state->context =<br> eglCreateContext (state->display, config, EGL_NO_CONTEXT, context_attributes);<br> assert (state->context != EGL_NO_CONTEXT);<br> </div> <div>//</div> <div>// Initialize GStreamer related resources.</div> <div>//</div> <div>state->gst_display = gst_gl_display_egl_new_with_egl_display (state->display);<br> state->gl_context =<br> gst_gl_context_new_wrapped (GST_GL_DISPLAY (state->gst_display),<br> (guintptr) state->context, GST_GL_PLATFORM_EGL, GST_GL_API_GLES2);<br> <br> GError *error = NULL;<br> if ( !gst_gl_display_create_context(GST_GL_DISPLAY(state->gst_display), state->gl_context, &state->newContext, &error) )<br> g_print("Failed to create new context\n");</div> <div><br> if ( !gst_gl_display_add_context(GST_GL_DISPLAY(state->gst_display), state->newContext))<br> g_print("Failed to add new context to display\n");</div> <div><br> </div> <div>} // init_ogl end.</div> <div><br> </div> <div><br> </div> <div>static void<br> sync_bus_call (GstBus * bus, GstMessage * msg, gpointer * data)<br> {<br> <br> APP_STATE_T *state = (APP_STATE_T *)data;<br> switch (GST_MESSAGE_TYPE (msg))<br> {<br> case GST_MESSAGE_NEED_CONTEXT:<br> {<br> const gchar *context_type;<br> gst_message_parse_context_type (msg, &context_type);<br> <br> GstContext *context = NULL;<br> if (g_strcmp0 (context_type, GST_GL_DISPLAY_CONTEXT_TYPE) == 0)<br> {<br> GstGLDisplay * gl_display = GST_GL_DISPLAY(state->gst_display);<br> context = gst_context_new (GST_GL_DISPLAY_CONTEXT_TYPE, TRUE);<br> gst_context_set_gl_display(context, gl_display);<br> gst_element_set_context (GST_ELEMENT(msg->src), context);<br> }<br> else if (g_strcmp0 (context_type, "gst.gl.app_context") == 0)<br> {<br> GstContext *context = gst_context_new("gst.gl.app_context", TRUE);<br> GstStructure *s = gst_context_writable_structure (context);<br> gst_structure_set (s, "context", GST_TYPE_GL_CONTEXT, state->gl_context, NULL);<br> gst_element_set_context(GST_ELEMENT(msg->src), context); <br> }<br> break;<br> }<br> default:<br> break;<br> }<br> } // sync_bus_call end<br> </div> <div><br> </div> <div><br> </div> <div> <div>I use need-data callback to create a buffer from texture_id and and push it in the appsrc:</div> <div><i>g_signal_connect (state->appsrc, "need-data", G_CALLBACK (</i> pushFrame <i>), state);</i></div> </div> <div><i><br> </i></div> <div>static bool pushFrame(..., APP_STATE_T * state)<br> {<br> // Wrap the texture into GstGLMemory<br> GstVideoInfo vinfo;<br> gst_video_info_set_format(&vinfo, GST_VIDEO_FORMAT_RGBA, state->screen_width, state->screen_height);<br> // Use state->newContext for allocator.<br> GstAllocator* allocator = GST_ALLOCATOR(gst_gl_memory_allocator_get_default(state->newContext));<br> GstGLVideoAllocationParams* params = gst_gl_video_allocation_params_new_wrapped_texture(<br> state->newContext, NULL, &vinfo, 0, NULL, GST_GL_TEXTURE_TARGET_2D, GST_GL_RGBA, state->tex,<br> NULL, 0);<br> <br> GstGLMemory* glMemory = GST_GL_MEMORY_CAST(gst_gl_base_memory_alloc(<br> GST_GL_BASE_MEMORY_ALLOCATOR_CAST(allocator), (GstGLAllocationParams*) params));<br> <br> gst_gl_allocation_params_free((GstGLAllocationParams *)params);<br> gst_object_unref(allocator);<br> <br> // Attach GstGLMemory object into buffer, timestamp the buffer and push it downstream<br> GstBuffer* buffer = gst_buffer_new();<br> gst_buffer_append_memory(buffer, GST_MEMORY_CAST(glMemory));<br> <br> GstVideoMeta * vmeta = gst_buffer_add_video_meta(buffer, GST_VIDEO_FRAME_FLAG_NONE, GST_VIDEO_FORMAT_RGBA, state->screen_width, state->screen_height);<br> <br> // Put timestamps into buffer<br> GST_BUFFER_PTS (buffer) = timestamp;<br> GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int (1, GST_SECOND, 2);<br> timestamp += GST_BUFFER_DURATION (buffer);<br> <br> GstFlowReturn ret;<br> g_signal_emit_by_name(state->appsrc, "push-buffer", buffer, &ret);<br> <br> if (ret != GST_FLOW_OK)<br> {<br> // Something wrong, stop pushing.<br> g_printerr("Something went wrong: Pushing buffer into appsrc is stopped.\n");<br> return false;<br> }<br> <br> return true;<br> } // pushFrame end<br> <br> </div> <div>Regards,</div> <div>Lusine</div> </div> <br> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">On Fri, Aug 6, 2021 at 8:59 AM Matthew Waters <<a href="mailto:ystreet00@gmail.com" target="_blank" moz-do-not-send="true">ystreet00@gmail.com</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div> Hi,<br> <br> <div>On 6/8/21 1:12 am, Lusine Hayrapetyan wrote:<br> </div> <blockquote type="cite"> <div dir="ltr">Dear Matt,<br> Thank you very much for your response. It helped me to understand that using 'wrapped' OpenGL context is a wrong direction to go :) <div><br> <i>WRT Suggestion 1:</i> <div>Do I understand correctly that I need to get local context from gstreamer and pass it to opengl rendering thread? (I have a rendering thread where I set OpenGL|ES context and screen stuff).</div> <div>GstGLContext* mContext = nullptr;<br> </div> <div>g_object_get(state->gldownload, "context", &mContext, NULL);<br> </div> <div>guintptr handle;<br> handle = gst_gl_context_get_gl_context(mContext); // is this correct?<br> state->context = (EGLContext)handle; // state->context is EGLContext type;<br> </div> <div>And then use state->context in OpenGL|ES?</div> </div> <div>Do I need to get and pass window and display from gstreamer to my rendering thread as well?</div> <div>Although my use scenario is different from this one - I need to pass context from OpenGL to Gstreamer.</div> </div> </blockquote> <br> You only need to retrieve or create a non-wrapped GstGLContext and use that for creating your textures that you are pushing into GStreamer. You don't need to use GStreamer's provided OpenGL context for anything else. Everything else in your sample remains the same. You may need to add a GstGLSyncMeta on your buffers you are pushing into GStreamer to provide the necessary synchronisation guarantees between the shared OpenGL contexts (application and GStreamer). On some platforms the window handle type and format may be important however in general on linux (X11/wayland) it doesn't really matter.<br> <br> You must not attempt to use GStreamer's OpenGL context as-is (using e.g. eglMakeCurrent() or anything of the like) from outside the GStreamer OpenGL context thread (as provided by the gst_gl_context_thread_add() API).<br> <br> <blockquote type="cite"> <div dir="ltr"> <div><i>WRT Suggestion 2:</i><br> </div> <div>gst_gl_display_create_context accepts <i>other_context </i>argument, should the <i>other_context </i>be the 'wrapped' context?</div> </div> </blockquote> <br> Yes. other_context is the GstGLContext that will be shared with the newly created GstGLContext.<br> <br> Side note, GStreamer cannot use any application-provided OpenGL context as-is due to the overhead of dealing with all the OpenGL state that may be changed behind GStreamer's back. This is why the OpenGL context sharing dance is required.<br> <br> Cheers<br> -Matt<br> <br> <blockquote type="cite"> <div dir="ltr"> <div>Best Regards,</div> <div>Lusine</div> </div> <br> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">On Thu, Aug 5, 2021 at 12:39 PM Matthew Waters <<a href="mailto:ystreet00@gmail.com" target="_blank" moz-do-not-send="true">ystreet00@gmail.com</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div> So, I think you almost have the correct sequence.<br> <br> Response inline.<br> <br> <div>On 5/8/21 1:04 am, Lusine Hayrapetyan via gstreamer-devel wrote:<br> </div> <blockquote type="cite"> <div dir="ltr">Hi Folks, <div>I'm struggling with the following issue and can't understand what I'm doing wrong.</div> <div>I need to pass opengl texture to the gstreamer pipeline.</div> <div>I have a rendering thread where I create opengl texture, the following objects created in this thread:</div> <div><i>EGLDisplay display;</i></div> <div><i>EGLContext context;</i><br> </div> <div><i><br> </i></div> <div>I create gstreamer pipeline in the main thread and as described in the following article sharing an X11 display and GstGLContext with the bus callback.</div> <div><a href="http://ystreet00.blogspot.com/2015/09/gstreamer-16-and-opengl-contexts.html" target="_blank" moz-do-not-send="true">http://ystreet00.blogspot.com/2015/09/gstreamer-16-and-opengl-contexts.html</a><br> </div> <div><br> </div> <div>GstGLDisplayEGL and GstGLContext are created in this way:</div> <div><i>GstGLDisplayEGL* gst_display = gst_gl_display_egl_new_with_egl_display (display);<br> </i></div> <div><i>GstGLContext *gl_context =<br> gst_gl_context_new_wrapped (GST_GL_DISPLAY (gst_display),<br> (guintptr) context, GST_GL_PLATFORM_EGL, GST_GL_API_GLES2);</i><br> </div> <div><br> </div> <div>The first element of my pipeline is appsrc:</div> <div><i>appsrc stream-type=0 emit-signals=1 format=3 caps=video/x-raw(memory:GLMemory), width=300, height=300, framerate=(fraction)20/1, format=(string)RGBA ! gldownload ! ...</i><br> </div> <div><br> </div> <div>I use need-data callback to create a buffer from texture_id and and push it in the appsrc:</div> <div><i>g_signal_connect (state->appsrc, "need-data", G_CALLBACK (</i> pushFrame <i>), state);</i></div> <div><i><br> </i></div> <div><i>bool pushFrame()</i><br> </div> <div><i>{<br> // Wrap the texture into GstGLMemory<br> GstVideoInfo vinfo;<br> gst_video_info_set_format(&vinfo, GST_VIDEO_FORMAT_RGBA, 300, 300);<br> <br> GstAllocator* allocator = GST_ALLOCATOR(gst_gl_memory_allocator_get_default(gl_context));<br> <br> GstGLVideoAllocationParams* params = gst_gl_video_allocation_params_new_wrapped_texture(<br> state->gl_context, NULL, &vinfo, 0, NULL, GST_GL_TEXTURE_TARGET_2D, GST_GL_RGBA, </i> texture_id <i>,<br> NULL, 0);<br> </i></div> </div> </blockquote> <br> The use of state->gl_context is probably your OpenGL context that has been wrapped from the application. This 'wrapped' OpenGL context has some limitations, one being that GStreamer cannot actually do a complete gst_gl_context_thread_add where the request is marshalled to an OpenGL-specific thread. This is what that critical is complaining about effectively.<br> <br> To do this properly, you would need to do one of two things:<br> 1. Retrieve the OpenGL context from the downstream gldownload element using either the 'context' property or using an appropriate GST_CONTEXT QUERY or the helper gst_gl_query_local_gl_context().<br> 2. Create your own GStreamer OpenGL context and add it to the GstGLDisplay using something like: <a href="https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/gst-libs/gst/gl/gstglbasefilter.c#L550-565" target="_blank" moz-do-not-send="true">https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/gst-libs/gst/gl/gstglbasefilter.c#L550-565</a>.<br> <br> Cheers<br> -Matt<br> <br> <blockquote type="cite"> <div dir="ltr"> <div><i><br> // The following line produces an error!!!<br> GstGLMemory* glMemory = GST_GL_MEMORY_CAST(gst_gl_base_memory_alloc(<br> GST_GL_BASE_MEMORY_ALLOCATOR_CAST(allocator), (GstGLAllocationParams*) params));<br> <br> gst_gl_allocation_params_free((GstGLAllocationParams *)params);<br> gst_object_unref(allocator);<br> <br> // Attach GstGLMemory object into buffer, timestamp the buffer and push it downstream<br> GstBuffer* buffer = gst_buffer_new();<br> gst_buffer_append_memory(buffer, GST_MEMORY_CAST(glMemory));<br> <br> // Put timestamps into buffer<br> GST_BUFFER_PTS (buffer) = timestamp;<br> GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int (1, GST_SECOND, 2);<br> <br> timestamp += GST_BUFFER_DURATION (buffer);<br> GstFlowReturn ret;<br> g_signal_emit_by_name(state->appsrc, "push-buffer", buffer, &ret);<br> <br> if (ret != GST_FLOW_OK)<br> {<br> // Something wrong, stop pushing.<br> g_printerr("Something went wrong: Pushing buffer into appsrc is stopped.\n");<br> return false;<br> }<br> <br> return true;<br> }<br> </i></div> <div><br> </div> <div>pushFrame produces the following error:</div> <div><font color="#ff0000">gst_gl_context_thread_add: assertion 'context->priv->active_thread == g_thread_self ()' failIed<br> </font></div> <div><font color="#ff0000"><br> </font></div> <div><font color="#000000">What am I doing wrong or how can push gpu texture to gstreamer?</font></div> <div><font color="#000000"><br> </font></div> <div><font color="#000000">Thanks,</font></div> <div><font color="#000000">Lusine</font></div> <div><font color="#000000"><br> </font></div> </div> </blockquote> <br> </div> </blockquote> </div> </blockquote> <br> </div> </blockquote> </div> </div> </blockquote> <br> </div> </blockquote> </div> </blockquote> <br> </div> </blockquote> </div> </blockquote> <br> </div> </blockquote> </div> </blockquote> <br> </div> </blockquote> </div> </blockquote> <br> </div> </blockquote> </div> </blockquote> <br> </body> </html>