<div dir="ltr">For limiting to a particular OpenGL variant I'm using the following environment variables, is it the same?<div><i> export GST_GL_WINDOW=x11<br> export GST_GL_PLATFORM=egl<br> export GST_GL_API=gles2</i><br></div><div><i><br></i></div><div>I've attached GST_DEBUG=gl*:7 output (gl_output.txt).</div><div>I see the following messages in the output:<br><i>0:00:03.857084617 24438 0x55d59121d400 INFO glwindow gstglwindow.c:278:gst_gl_window_new: creating a window, user choice:x11<br>0:00:03.857117499 24438 0x55d59121d400 INFO glwindow gstglwindow_x11.c:137:gst_gl_window_x11_new: Wrong display type 32 for this window type 1<br>0:00:03.857144332 24438 0x55d59121d400 WARN glwindow gstglwindow.c:324:gst_gl_window_new: Could not create window. user specified x11, creating dummy window<br>0:00:03.857277761 24438 0x55d59121d400 DEBUG gldisplay gstgldisplay.c:579:gst_gl_display_create_window:<gldisplayegl0> Adding window <gldummywindow0> (0x55d5913809d0) to internal list<br>0:00:03.857311159 24438 0x55d59121d400 DEBUG glcontext gstglcontext.c:948:gst_gl_context_set_window:<glcontextegl0> window:<gldummywindow0></i><br></div><div><i><br></i></div><div>Is this a critical issue? Can this cause a texture reading issue?</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Aug 10, 2021 at 10:21 AM Matthew Waters <<a href="mailto:ystreet00@gmail.com">ystreet00@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
Ah, if you want to limit to a particular OpenGL variant, you need
to also call gst_gl_display_filter_gl_api() with that variant.<br>
<br>
GST_DEBUG=gl*:7 will output a whole bunch of information about what
is generated and tried in all of this respect.<br>
<br>
Cheers<br>
-Matt<br>
<br>
<div>On 9/8/21 12:58 am, Lusine Hayrapetyan
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">Hi Matthew,<br>
Seems the second approach fits to my use case- I need to push
textures which are created in opengles & egl( it means not
in gstreamer context) to gstreamer.
<div>I've changed my code to use
<code>gst_gl_display_create_context & </code>
<code><span>gst_gl_display_add_context <font face="arial,
sans-serif">but still can't read textures in
gstreamer, gstreamer prudeces the following errors
after I push buffer to appsrc:</font></span></code></div>
<div><code>(testegl1:24602): GStreamer-CRITICAL **:
13:02:55.568: gst_debug_log_valist: assertion 'category !=
NULL' failed<br>
0:00:14.039444188 24602 0x5611bc356d90 WARN
glbasetexture gstglmemory.c:401:gst_gl_memory_read_pixels:
Could not create framebuffer to read pixels for memory
0x7f8cf0017ac0<br>
0:00:14.039480909 24602 0x5611bc356d90 WARN
glbasememory
gstglbasememory.c:585:gst_gl_base_memory_memcpy: could not
read map source memory 0x7f8cf0017ac0<br>
0:00:14.039505741 24602 0x5611bc356d90 WARN
glmemory gstglmemorypbo.c:592:_gl_mem_copy: Could not
copy GL Memory<br>
0:00:14.039779268 24602 0x5611bc4f14f0 ERROR
videometa gstvideometa.c:247:default_map: cannot map
memory range 0-1<br>
0:00:14.039846056 24602 0x5611bc4f14f0 ERROR
default video-frame.c:168:gst_video_frame_map_id: failed
to map video frame plane 0<br>
0:00:14.039891314 24602 0x5611bc4f14f0 WARN
videofilter
gstvideofilter.c:297:gst_video_filter_transform:<videoconvert0>
warning: invalid video buffer received<span><font face="arial, sans-serif"><br>
</font></span></code></div>
<div><code><span><font face="arial, sans-serif"><br>
</font></span></code></div>
<div><code><span><font face="arial, sans-serif"><br>
</font></span></code></div>
<div><code><span><font face="arial, sans-serif">This is how I
implemented it, did I misunderstand something? </font></span></code></div>
<div><code><span><font face="arial, sans-serif"><br>
</font></span></code></div>
<div>I have rendering thread where I initialize opengles
context and create wrapped and the new contexts.</div>
<div>//</div>
<div>// Description: Sets the display, OpenGL|ES context and
screen stuff</div>
<div>// Created GstGLContext s - wrapped
(gst_gl_context_new_wrapped) and new
context(gst_gl_display_create_context)</div>
<div>//</div>
<div>static void<br>
init_ogl (APP_STATE_T * state)<br>
</div>
<div>{</div>
<div>...</div>
/* get an EGL display connection */<br>
state->display = eglGetDisplay (EGL_DEFAULT_DISPLAY);<br>
assert (state->display != EGL_NO_DISPLAY);<br>
/* initialize the EGL display connection */<br>
result = eglInitialize (state->display, NULL, NULL);<br>
<div> assert (EGL_FALSE != result); </div>
<div><br>
</div>
<div>/* create an EGL rendering context */<br>
state->context =<br>
eglCreateContext (state->display, config,
EGL_NO_CONTEXT, context_attributes);<br>
assert (state->context != EGL_NO_CONTEXT);<br>
</div>
<div>//</div>
<div>// Initialize GStreamer related resources.</div>
<div>//</div>
<div>state->gst_display =
gst_gl_display_egl_new_with_egl_display (state->display);<br>
state->gl_context =<br>
gst_gl_context_new_wrapped (GST_GL_DISPLAY
(state->gst_display),<br>
(guintptr) state->context, GST_GL_PLATFORM_EGL,
GST_GL_API_GLES2);<br>
<br>
GError *error = NULL;<br>
if (
!gst_gl_display_create_context(GST_GL_DISPLAY(state->gst_display),
state->gl_context, &state->newContext, &error)
)<br>
g_print("Failed to create new context\n");</div>
<div><br>
if (
!gst_gl_display_add_context(GST_GL_DISPLAY(state->gst_display),
state->newContext))<br>
g_print("Failed to add new context to display\n");</div>
<div><br>
</div>
<div>} // init_ogl end.</div>
<div><br>
</div>
<div><br>
</div>
<div>static void<br>
sync_bus_call (GstBus * bus, GstMessage * msg, gpointer *
data)<br>
{<br>
<br>
APP_STATE_T *state = (APP_STATE_T *)data;<br>
switch (GST_MESSAGE_TYPE (msg))<br>
{<br>
case GST_MESSAGE_NEED_CONTEXT:<br>
{<br>
const gchar *context_type;<br>
gst_message_parse_context_type (msg,
&context_type);<br>
<br>
GstContext *context = NULL;<br>
if (g_strcmp0 (context_type,
GST_GL_DISPLAY_CONTEXT_TYPE) == 0)<br>
{<br>
GstGLDisplay * gl_display =
GST_GL_DISPLAY(state->gst_display);<br>
context = gst_context_new
(GST_GL_DISPLAY_CONTEXT_TYPE, TRUE);<br>
gst_context_set_gl_display(context,
gl_display);<br>
gst_element_set_context
(GST_ELEMENT(msg->src), context);<br>
}<br>
else if (g_strcmp0 (context_type,
"gst.gl.app_context") == 0)<br>
{<br>
GstContext *context =
gst_context_new("gst.gl.app_context", TRUE);<br>
GstStructure *s =
gst_context_writable_structure (context);<br>
gst_structure_set (s, "context",
GST_TYPE_GL_CONTEXT, state->gl_context, NULL);<br>
gst_element_set_context(GST_ELEMENT(msg->src),
context); <br>
}<br>
break;<br>
}<br>
default:<br>
break;<br>
}<br>
} // sync_bus_call end<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div>I use need-data callback to create a buffer from
texture_id and and push it in the appsrc:</div>
<div><i>g_signal_connect (state->appsrc, "need-data",
G_CALLBACK (</i> pushFrame <i>), state);</i></div>
</div>
<div><i><br>
</i></div>
<div>static bool pushFrame(..., APP_STATE_T * state)<br>
{<br>
// Wrap the texture into GstGLMemory<br>
GstVideoInfo vinfo;<br>
gst_video_info_set_format(&vinfo,
GST_VIDEO_FORMAT_RGBA, state->screen_width,
state->screen_height);<br>
// Use
state->newContext for allocator.<br>
GstAllocator* allocator =
GST_ALLOCATOR(gst_gl_memory_allocator_get_default(state->newContext));<br>
GstGLVideoAllocationParams* params =
gst_gl_video_allocation_params_new_wrapped_texture(<br>
state->newContext, NULL, &vinfo, 0, NULL,
GST_GL_TEXTURE_TARGET_2D, GST_GL_RGBA, state->tex,<br>
NULL, 0);<br>
<br>
GstGLMemory* glMemory =
GST_GL_MEMORY_CAST(gst_gl_base_memory_alloc(<br>
GST_GL_BASE_MEMORY_ALLOCATOR_CAST(allocator),
(GstGLAllocationParams*) params));<br>
<br>
gst_gl_allocation_params_free((GstGLAllocationParams
*)params);<br>
gst_object_unref(allocator);<br>
<br>
// Attach GstGLMemory object into buffer, timestamp the
buffer and push it downstream<br>
GstBuffer* buffer = gst_buffer_new();<br>
gst_buffer_append_memory(buffer,
GST_MEMORY_CAST(glMemory));<br>
<br>
GstVideoMeta * vmeta = gst_buffer_add_video_meta(buffer,
GST_VIDEO_FRAME_FLAG_NONE, GST_VIDEO_FORMAT_RGBA,
state->screen_width, state->screen_height);<br>
<br>
// Put timestamps into buffer<br>
GST_BUFFER_PTS (buffer) = timestamp;<br>
GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int
(1, GST_SECOND, 2);<br>
timestamp += GST_BUFFER_DURATION (buffer);<br>
<br>
GstFlowReturn ret;<br>
g_signal_emit_by_name(state->appsrc, "push-buffer",
buffer, &ret);<br>
<br>
if (ret != GST_FLOW_OK)<br>
{<br>
// Something wrong, stop pushing.<br>
g_printerr("Something went wrong: Pushing buffer
into appsrc is stopped.\n");<br>
return false;<br>
}<br>
<br>
return true;<br>
} // pushFrame end<br>
<br>
</div>
<div>Regards,</div>
<div>Lusine</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Aug 6, 2021 at 8:59
AM Matthew Waters <<a href="mailto:ystreet00@gmail.com" target="_blank">ystreet00@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> Hi,<br>
<br>
<div>On 6/8/21 1:12 am, Lusine Hayrapetyan wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Dear Matt,<br>
Thank you very much for your response. It helped me to
understand that using 'wrapped' OpenGL context is a
wrong direction to go :)
<div><br>
<i>WRT Suggestion 1:</i>
<div>Do I understand correctly that I need to get
local context from gstreamer and pass it to opengl
rendering thread? (I have a rendering thread where
I set OpenGL|ES context and screen stuff).</div>
<div>GstGLContext* mContext = nullptr;<br>
</div>
<div>g_object_get(state->gldownload, "context",
&mContext, NULL);<br>
</div>
<div>guintptr handle;<br>
handle = gst_gl_context_get_gl_context(mContext);
// is this correct?<br>
state->context = (EGLContext)handle; //
state->context is EGLContext type;<br>
</div>
<div>And then use state->context in OpenGL|ES?</div>
</div>
<div>Do I need to get and pass window and display
from gstreamer to my rendering thread as well?</div>
<div>Although my use scenario is different from this
one - I need to pass context from OpenGL to
Gstreamer.</div>
</div>
</blockquote>
<br>
You only need to retrieve or create a non-wrapped
GstGLContext and use that for creating your textures that
you are pushing into GStreamer. You don't need to use
GStreamer's provided OpenGL context for anything else.
Everything else in your sample remains the same. You may
need to add a GstGLSyncMeta on your buffers you are
pushing into GStreamer to provide the necessary
synchronisation guarantees between the shared OpenGL
contexts (application and GStreamer). On some platforms
the window handle type and format may be important however
in general on linux (X11/wayland) it doesn't really
matter.<br>
<br>
You must not attempt to use GStreamer's OpenGL context
as-is (using e.g. eglMakeCurrent() or anything of the
like) from outside the GStreamer OpenGL context thread (as
provided by the gst_gl_context_thread_add() API).<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div><i>WRT Suggestion 2:</i><br>
</div>
<div>gst_gl_display_create_context accepts <i>other_context
</i>argument, should the <i>other_context </i>be the
'wrapped' context?</div>
</div>
</blockquote>
<br>
Yes. other_context is the GstGLContext that will be
shared with the newly created GstGLContext.<br>
<br>
Side note, GStreamer cannot use any application-provided
OpenGL context as-is due to the overhead of dealing with
all the OpenGL state that may be changed behind
GStreamer's back. This is why the OpenGL context sharing
dance is required.<br>
<br>
Cheers<br>
-Matt<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>Best Regards,</div>
<div>Lusine</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Aug 5, 2021
at 12:39 PM Matthew Waters <<a href="mailto:ystreet00@gmail.com" target="_blank">ystreet00@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> So, I think you almost have the correct
sequence.<br>
<br>
Response inline.<br>
<br>
<div>On 5/8/21 1:04 am, Lusine Hayrapetyan via
gstreamer-devel wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Folks,
<div>I'm struggling with the following issue
and can't understand what I'm doing wrong.</div>
<div>I need to pass opengl texture to the
gstreamer pipeline.</div>
<div>I have a rendering thread where I create
opengl texture, the following objects
created in this thread:</div>
<div><i>EGLDisplay display;</i></div>
<div><i>EGLContext context;</i><br>
</div>
<div><i><br>
</i></div>
<div>I create gstreamer pipeline in the main
thread and as described in the following
article sharing an X11 display and
GstGLContext with the bus callback.</div>
<div><a href="http://ystreet00.blogspot.com/2015/09/gstreamer-16-and-opengl-contexts.html" target="_blank">http://ystreet00.blogspot.com/2015/09/gstreamer-16-and-opengl-contexts.html</a><br>
</div>
<div><br>
</div>
<div>GstGLDisplayEGL and GstGLContext are
created in this way:</div>
<div><i>GstGLDisplayEGL* gst_display =
gst_gl_display_egl_new_with_egl_display
(display);<br>
</i></div>
<div><i>GstGLContext *gl_context =<br>
gst_gl_context_new_wrapped
(GST_GL_DISPLAY (gst_display),<br>
(guintptr) context,
GST_GL_PLATFORM_EGL, GST_GL_API_GLES2);</i><br>
</div>
<div><br>
</div>
<div>The first element of my pipeline is
appsrc:</div>
<div><i>appsrc stream-type=0 emit-signals=1
format=3
caps=video/x-raw(memory:GLMemory), width=300,
height=300, framerate=(fraction)20/1,
format=(string)RGBA ! gldownload ! ...</i><br>
</div>
<div><br>
</div>
<div>I use need-data callback to create a
buffer from texture_id and and push it in
the appsrc:</div>
<div><i>g_signal_connect (state->appsrc,
"need-data", G_CALLBACK (</i> pushFrame <i>),
state);</i></div>
<div><i><br>
</i></div>
<div><i>bool pushFrame()</i><br>
</div>
<div><i>{<br>
// Wrap the texture into GstGLMemory<br>
GstVideoInfo vinfo;<br>
gst_video_info_set_format(&vinfo,
GST_VIDEO_FORMAT_RGBA, 300, 300);<br>
<br>
GstAllocator* allocator =
GST_ALLOCATOR(gst_gl_memory_allocator_get_default(gl_context));<br>
<br>
GstGLVideoAllocationParams* params =
gst_gl_video_allocation_params_new_wrapped_texture(<br>
state->gl_context, NULL,
&vinfo, 0, NULL,
GST_GL_TEXTURE_TARGET_2D, GST_GL_RGBA, </i>
texture_id <i>,<br>
NULL, 0);<br>
</i></div>
</div>
</blockquote>
<br>
The use of state->gl_context is probably your
OpenGL context that has been wrapped from the
application. This 'wrapped' OpenGL context has
some limitations, one being that GStreamer cannot
actually do a complete gst_gl_context_thread_add
where the request is marshalled to an
OpenGL-specific thread. This is what that
critical is complaining about effectively.<br>
<br>
To do this properly, you would need to do one of
two things:<br>
1. Retrieve the OpenGL context from the downstream
gldownload element using either the 'context'
property or using an appropriate GST_CONTEXT QUERY
or the helper gst_gl_query_local_gl_context().<br>
2. Create your own GStreamer OpenGL context and
add it to the GstGLDisplay using something like: <a href="https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/gst-libs/gst/gl/gstglbasefilter.c#L550-565" target="_blank">https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/gst-libs/gst/gl/gstglbasefilter.c#L550-565</a>.<br>
<br>
Cheers<br>
-Matt<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div><i><br>
// The following line produces an
error!!!<br>
GstGLMemory* glMemory =
GST_GL_MEMORY_CAST(gst_gl_base_memory_alloc(<br>
GST_GL_BASE_MEMORY_ALLOCATOR_CAST(allocator),
(GstGLAllocationParams*) params));<br>
<br>
gst_gl_allocation_params_free((GstGLAllocationParams
*)params);<br>
gst_object_unref(allocator);<br>
<br>
// Attach GstGLMemory object into
buffer, timestamp the buffer and push it
downstream<br>
GstBuffer* buffer = gst_buffer_new();<br>
gst_buffer_append_memory(buffer,
GST_MEMORY_CAST(glMemory));<br>
<br>
// Put timestamps into buffer<br>
GST_BUFFER_PTS (buffer) = timestamp;<br>
GST_BUFFER_DURATION (buffer) =
gst_util_uint64_scale_int (1, GST_SECOND,
2);<br>
<br>
timestamp += GST_BUFFER_DURATION
(buffer);<br>
GstFlowReturn ret;<br>
g_signal_emit_by_name(state->appsrc,
"push-buffer", buffer, &ret);<br>
<br>
if (ret != GST_FLOW_OK)<br>
{<br>
// Something wrong, stop pushing.<br>
g_printerr("Something went wrong:
Pushing buffer into appsrc is
stopped.\n");<br>
return false;<br>
}<br>
<br>
return true;<br>
}<br>
</i></div>
<div><br>
</div>
<div>pushFrame produces the following error:</div>
<div><font color="#ff0000">gst_gl_context_thread_add:
assertion
'context->priv->active_thread ==
g_thread_self ()' failIed<br>
</font></div>
<div><font color="#ff0000"><br>
</font></div>
<div><font color="#000000">What am I doing
wrong or how can push gpu texture to
gstreamer?</font></div>
<div><font color="#000000"><br>
</font></div>
<div><font color="#000000">Thanks,</font></div>
<div><font color="#000000">Lusine</font></div>
<div><font color="#000000"><br>
</font></div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
</div>
</blockquote></div>