<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
Yes, that is a critical error for X11. X11 would require access to
an X11 Display which must be provided by your application in order
share OpenGL resources. You may also not have X11 support built and
you would have to check your build configuration for that.
gstglconfig.h would contain information on what libgstgl-1.0 was
built with.<br>
<br>
It looks like it's picking a raw EGLDisplay instead. This was
potentially an issue in previous versions of GStreamer that has been
fixed in master by <a moz-do-not-send="true"
href="https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1154">https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1154</a>
and <a moz-do-not-send="true"
href="https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1169">https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1169</a>.<br>
<br>
Cheers<br>
-Matt<br>
<br>
<div class="moz-cite-prefix">On 12/8/21 1:50 am, Lusine Hayrapetyan
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAO4TkMiz9xBous6pnNA=MZS3NbFzWRjHHfr63XbRwep5nnZ8qA@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">For limiting to a particular OpenGL variant I'm
using the following environment variables, is it the same?
<div><i> export GST_GL_WINDOW=x11<br>
export GST_GL_PLATFORM=egl<br>
export GST_GL_API=gles2</i><br>
</div>
<div><i><br>
</i></div>
<div>I've attached GST_DEBUG=gl*:7 output (gl_output.txt).</div>
<div>I see the following messages in the output:<br>
<i>0:00:03.857084617 24438 0x55d59121d400 INFO
glwindow gstglwindow.c:278:gst_gl_window_new: creating a
window, user choice:x11<br>
0:00:03.857117499 24438 0x55d59121d400 INFO
glwindow gstglwindow_x11.c:137:gst_gl_window_x11_new: Wrong
display type 32 for this window type 1<br>
0:00:03.857144332 24438 0x55d59121d400 WARN
glwindow gstglwindow.c:324:gst_gl_window_new: Could not
create window. user specified x11, creating dummy window<br>
0:00:03.857277761 24438 0x55d59121d400 DEBUG
gldisplay
gstgldisplay.c:579:gst_gl_display_create_window:<gldisplayegl0>
Adding window <gldummywindow0> (0x55d5913809d0) to
internal list<br>
0:00:03.857311159 24438 0x55d59121d400 DEBUG
glcontext
gstglcontext.c:948:gst_gl_context_set_window:<glcontextegl0>
window:<gldummywindow0></i><br>
</div>
<div><i><br>
</i></div>
<div>Is this a critical issue? Can this cause a texture reading
issue?</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Aug 10, 2021 at 10:21
AM Matthew Waters <<a href="mailto:ystreet00@gmail.com"
moz-do-not-send="true">ystreet00@gmail.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> Ah, if you want to limit to a particular OpenGL
variant, you need to also call
gst_gl_display_filter_gl_api() with that variant.<br>
<br>
GST_DEBUG=gl*:7 will output a whole bunch of information
about what is generated and tried in all of this respect.<br>
<br>
Cheers<br>
-Matt<br>
<br>
<div>On 9/8/21 12:58 am, Lusine Hayrapetyan wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">Hi Matthew,<br>
Seems the second approach fits to my use case- I need
to push textures which are created in opengles &
egl( it means not in gstreamer context) to gstreamer.
<div>I've changed my code to use <code>gst_gl_display_create_context
& </code> <code><span>gst_gl_display_add_context
<font face="arial, sans-serif">but still can't
read textures in gstreamer, gstreamer prudeces
the following errors after I push buffer to
appsrc:</font></span></code></div>
<div><code>(testegl1:24602): GStreamer-CRITICAL **:
13:02:55.568: gst_debug_log_valist: assertion
'category != NULL' failed<br>
0:00:14.039444188 24602 0x5611bc356d90 WARN
glbasetexture
gstglmemory.c:401:gst_gl_memory_read_pixels: Could
not create framebuffer to read pixels for memory
0x7f8cf0017ac0<br>
0:00:14.039480909 24602 0x5611bc356d90 WARN
glbasememory
gstglbasememory.c:585:gst_gl_base_memory_memcpy:
could not read map source memory 0x7f8cf0017ac0<br>
0:00:14.039505741 24602 0x5611bc356d90 WARN
glmemory
gstglmemorypbo.c:592:_gl_mem_copy: Could not copy
GL Memory<br>
0:00:14.039779268 24602 0x5611bc4f14f0 ERROR
videometa gstvideometa.c:247:default_map:
cannot map memory range 0-1<br>
0:00:14.039846056 24602 0x5611bc4f14f0 ERROR
default
video-frame.c:168:gst_video_frame_map_id: failed
to map video frame plane 0<br>
0:00:14.039891314 24602 0x5611bc4f14f0 WARN
videofilter
gstvideofilter.c:297:gst_video_filter_transform:<videoconvert0>
warning: invalid video buffer received<span><font
face="arial, sans-serif"><br>
</font></span></code></div>
<div><code><span><font face="arial, sans-serif"><br>
</font></span></code></div>
<div><code><span><font face="arial, sans-serif"><br>
</font></span></code></div>
<div><code><span><font face="arial, sans-serif">This
is how I implemented it, did I misunderstand
something? </font></span></code></div>
<div><code><span><font face="arial, sans-serif"><br>
</font></span></code></div>
<div>I have rendering thread where I initialize
opengles context and create wrapped and the new
contexts.</div>
<div>//</div>
<div>// Description: Sets the display, OpenGL|ES
context and screen stuff</div>
<div>// Created GstGLContext s - wrapped
(gst_gl_context_new_wrapped) and new
context(gst_gl_display_create_context)</div>
<div>//</div>
<div>static void<br>
init_ogl (APP_STATE_T * state)<br>
</div>
<div>{</div>
<div>...</div>
/* get an EGL display connection */<br>
state->display = eglGetDisplay
(EGL_DEFAULT_DISPLAY);<br>
assert (state->display != EGL_NO_DISPLAY);<br>
/* initialize the EGL display connection */<br>
result = eglInitialize (state->display, NULL,
NULL);<br>
<div> assert (EGL_FALSE != result); </div>
<div><br>
</div>
<div>/* create an EGL rendering context */<br>
state->context =<br>
eglCreateContext (state->display, config,
EGL_NO_CONTEXT, context_attributes);<br>
assert (state->context != EGL_NO_CONTEXT);<br>
</div>
<div>//</div>
<div>// Initialize GStreamer related resources.</div>
<div>//</div>
<div>state->gst_display =
gst_gl_display_egl_new_with_egl_display
(state->display);<br>
state->gl_context =<br>
gst_gl_context_new_wrapped (GST_GL_DISPLAY
(state->gst_display),<br>
(guintptr) state->context,
GST_GL_PLATFORM_EGL, GST_GL_API_GLES2);<br>
<br>
GError *error = NULL;<br>
if (
!gst_gl_display_create_context(GST_GL_DISPLAY(state->gst_display),
state->gl_context, &state->newContext,
&error) )<br>
g_print("Failed to create new context\n");</div>
<div><br>
if (
!gst_gl_display_add_context(GST_GL_DISPLAY(state->gst_display),
state->newContext))<br>
g_print("Failed to add new context to
display\n");</div>
<div><br>
</div>
<div>} // init_ogl end.</div>
<div><br>
</div>
<div><br>
</div>
<div>static void<br>
sync_bus_call (GstBus * bus, GstMessage * msg,
gpointer * data)<br>
{<br>
<br>
APP_STATE_T *state = (APP_STATE_T *)data;<br>
switch (GST_MESSAGE_TYPE (msg))<br>
{<br>
case GST_MESSAGE_NEED_CONTEXT:<br>
{<br>
const gchar *context_type;<br>
gst_message_parse_context_type (msg,
&context_type);<br>
<br>
GstContext *context = NULL;<br>
if (g_strcmp0 (context_type,
GST_GL_DISPLAY_CONTEXT_TYPE) == 0)<br>
{<br>
GstGLDisplay * gl_display =
GST_GL_DISPLAY(state->gst_display);<br>
context = gst_context_new
(GST_GL_DISPLAY_CONTEXT_TYPE, TRUE);<br>
gst_context_set_gl_display(context,
gl_display);<br>
gst_element_set_context
(GST_ELEMENT(msg->src), context);<br>
}<br>
else if (g_strcmp0 (context_type,
"gst.gl.app_context") == 0)<br>
{<br>
GstContext *context =
gst_context_new("gst.gl.app_context", TRUE);<br>
GstStructure *s =
gst_context_writable_structure (context);<br>
gst_structure_set (s, "context",
GST_TYPE_GL_CONTEXT, state->gl_context, NULL);<br>
gst_element_set_context(GST_ELEMENT(msg->src),
context); <br>
}<br>
break;<br>
}<br>
default:<br>
break;<br>
}<br>
} // sync_bus_call end<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div>I use need-data callback to create a buffer
from texture_id and and push it in the appsrc:</div>
<div><i>g_signal_connect (state->appsrc,
"need-data", G_CALLBACK (</i> pushFrame <i>),
state);</i></div>
</div>
<div><i><br>
</i></div>
<div>static bool pushFrame(..., APP_STATE_T * state)<br>
{<br>
// Wrap the texture into GstGLMemory<br>
GstVideoInfo vinfo;<br>
gst_video_info_set_format(&vinfo,
GST_VIDEO_FORMAT_RGBA, state->screen_width,
state->screen_height);<br>
// Use state->newContext for allocator.<br>
GstAllocator* allocator =
GST_ALLOCATOR(gst_gl_memory_allocator_get_default(state->newContext));<br>
GstGLVideoAllocationParams* params =
gst_gl_video_allocation_params_new_wrapped_texture(<br>
state->newContext, NULL, &vinfo, 0,
NULL, GST_GL_TEXTURE_TARGET_2D, GST_GL_RGBA,
state->tex,<br>
NULL, 0);<br>
<br>
GstGLMemory* glMemory =
GST_GL_MEMORY_CAST(gst_gl_base_memory_alloc(<br>
GST_GL_BASE_MEMORY_ALLOCATOR_CAST(allocator),
(GstGLAllocationParams*) params));<br>
<br>
gst_gl_allocation_params_free((GstGLAllocationParams
*)params);<br>
gst_object_unref(allocator);<br>
<br>
// Attach GstGLMemory object into buffer,
timestamp the buffer and push it downstream<br>
GstBuffer* buffer = gst_buffer_new();<br>
gst_buffer_append_memory(buffer,
GST_MEMORY_CAST(glMemory));<br>
<br>
GstVideoMeta * vmeta =
gst_buffer_add_video_meta(buffer,
GST_VIDEO_FRAME_FLAG_NONE, GST_VIDEO_FORMAT_RGBA,
state->screen_width, state->screen_height);<br>
<br>
// Put timestamps into buffer<br>
GST_BUFFER_PTS (buffer) = timestamp;<br>
GST_BUFFER_DURATION (buffer) =
gst_util_uint64_scale_int (1, GST_SECOND, 2);<br>
timestamp += GST_BUFFER_DURATION (buffer);<br>
<br>
GstFlowReturn ret;<br>
g_signal_emit_by_name(state->appsrc,
"push-buffer", buffer, &ret);<br>
<br>
if (ret != GST_FLOW_OK)<br>
{<br>
// Something wrong, stop pushing.<br>
g_printerr("Something went wrong: Pushing
buffer into appsrc is stopped.\n");<br>
return false;<br>
}<br>
<br>
return true;<br>
} // pushFrame end<br>
<br>
</div>
<div>Regards,</div>
<div>Lusine</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Aug 6, 2021
at 8:59 AM Matthew Waters <<a
href="mailto:ystreet00@gmail.com" target="_blank"
moz-do-not-send="true">ystreet00@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px
0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div> Hi,<br>
<br>
<div>On 6/8/21 1:12 am, Lusine Hayrapetyan wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Dear Matt,<br>
Thank you very much for your response. It
helped me to understand that using 'wrapped'
OpenGL context is a wrong direction to go :)
<div><br>
<i>WRT Suggestion 1:</i>
<div>Do I understand correctly that I need
to get local context from gstreamer and
pass it to opengl rendering thread? (I
have a rendering thread where I set
OpenGL|ES context and screen stuff).</div>
<div>GstGLContext* mContext = nullptr;<br>
</div>
<div>g_object_get(state->gldownload,
"context", &mContext, NULL);<br>
</div>
<div>guintptr handle;<br>
handle =
gst_gl_context_get_gl_context(mContext);
// is this correct?<br>
state->context = (EGLContext)handle; //
state->context is EGLContext type;<br>
</div>
<div>And then use state->context in
OpenGL|ES?</div>
</div>
<div>Do I need to get and pass window and
display from gstreamer to my rendering
thread as well?</div>
<div>Although my use scenario is different
from this one - I need to pass context from
OpenGL to Gstreamer.</div>
</div>
</blockquote>
<br>
You only need to retrieve or create a non-wrapped
GstGLContext and use that for creating your
textures that you are pushing into GStreamer. You
don't need to use GStreamer's provided OpenGL
context for anything else. Everything else in
your sample remains the same. You may need to add
a GstGLSyncMeta on your buffers you are pushing
into GStreamer to provide the necessary
synchronisation guarantees between the shared
OpenGL contexts (application and GStreamer). On
some platforms the window handle type and format
may be important however in general on linux
(X11/wayland) it doesn't really matter.<br>
<br>
You must not attempt to use GStreamer's OpenGL
context as-is (using e.g. eglMakeCurrent() or
anything of the like) from outside the GStreamer
OpenGL context thread (as provided by the
gst_gl_context_thread_add() API).<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div><i>WRT Suggestion 2:</i><br>
</div>
<div>gst_gl_display_create_context accepts <i>other_context
</i>argument, should the <i>other_context </i>be
the 'wrapped' context?</div>
</div>
</blockquote>
<br>
Yes. other_context is the GstGLContext that will
be shared with the newly created GstGLContext.<br>
<br>
Side note, GStreamer cannot use any
application-provided OpenGL context as-is due to
the overhead of dealing with all the OpenGL state
that may be changed behind GStreamer's back. This
is why the OpenGL context sharing dance is
required.<br>
<br>
Cheers<br>
-Matt<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>Best Regards,</div>
<div>Lusine</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Aug
5, 2021 at 12:39 PM Matthew Waters <<a
href="mailto:ystreet00@gmail.com"
target="_blank" moz-do-not-send="true">ystreet00@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div> So, I think you almost have the
correct sequence.<br>
<br>
Response inline.<br>
<br>
<div>On 5/8/21 1:04 am, Lusine Hayrapetyan
via gstreamer-devel wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Folks,
<div>I'm struggling with the following
issue and can't understand what I'm
doing wrong.</div>
<div>I need to pass opengl texture to
the gstreamer pipeline.</div>
<div>I have a rendering thread where I
create opengl texture, the following
objects created in this thread:</div>
<div><i>EGLDisplay display;</i></div>
<div><i>EGLContext context;</i><br>
</div>
<div><i><br>
</i></div>
<div>I create gstreamer pipeline in
the main thread and as described in
the following article sharing an X11
display and GstGLContext with the
bus callback.</div>
<div><a
href="http://ystreet00.blogspot.com/2015/09/gstreamer-16-and-opengl-contexts.html"
target="_blank"
moz-do-not-send="true">http://ystreet00.blogspot.com/2015/09/gstreamer-16-and-opengl-contexts.html</a><br>
</div>
<div><br>
</div>
<div>GstGLDisplayEGL and GstGLContext
are created in this way:</div>
<div><i>GstGLDisplayEGL* gst_display =
gst_gl_display_egl_new_with_egl_display (display);<br>
</i></div>
<div><i>GstGLContext *gl_context =<br>
gst_gl_context_new_wrapped
(GST_GL_DISPLAY (gst_display),<br>
(guintptr) context,
GST_GL_PLATFORM_EGL,
GST_GL_API_GLES2);</i><br>
</div>
<div><br>
</div>
<div>The first element of my pipeline
is appsrc:</div>
<div><i>appsrc stream-type=0
emit-signals=1 format=3
caps=video/x-raw(memory:GLMemory), width=300,
height=300,
framerate=(fraction)20/1,
format=(string)RGBA ! gldownload !
...</i><br>
</div>
<div><br>
</div>
<div>I use need-data callback to
create a buffer from texture_id and
and push it in the appsrc:</div>
<div><i>g_signal_connect
(state->appsrc, "need-data",
G_CALLBACK (</i> pushFrame <i>),
state);</i></div>
<div><i><br>
</i></div>
<div><i>bool pushFrame()</i><br>
</div>
<div><i>{<br>
// Wrap the texture into
GstGLMemory<br>
GstVideoInfo vinfo;<br>
gst_video_info_set_format(&vinfo,
GST_VIDEO_FORMAT_RGBA, 300, 300);<br>
<br>
GstAllocator* allocator =
GST_ALLOCATOR(gst_gl_memory_allocator_get_default(gl_context));<br>
<br>
GstGLVideoAllocationParams*
params =
gst_gl_video_allocation_params_new_wrapped_texture(<br>
state->gl_context, NULL,
&vinfo, 0, NULL,
GST_GL_TEXTURE_TARGET_2D,
GST_GL_RGBA, </i> texture_id <i>,<br>
NULL, 0);<br>
</i></div>
</div>
</blockquote>
<br>
The use of state->gl_context is
probably your OpenGL context that has been
wrapped from the application. This
'wrapped' OpenGL context has some
limitations, one being that GStreamer
cannot actually do a complete
gst_gl_context_thread_add where the
request is marshalled to an
OpenGL-specific thread. This is what that
critical is complaining about effectively.<br>
<br>
To do this properly, you would need to do
one of two things:<br>
1. Retrieve the OpenGL context from the
downstream gldownload element using either
the 'context' property or using an
appropriate GST_CONTEXT QUERY or the
helper gst_gl_query_local_gl_context().<br>
2. Create your own GStreamer OpenGL
context and add it to the GstGLDisplay
using something like: <a
href="https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/gst-libs/gst/gl/gstglbasefilter.c#L550-565"
target="_blank" moz-do-not-send="true">https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/gst-libs/gst/gl/gstglbasefilter.c#L550-565</a>.<br>
<br>
Cheers<br>
-Matt<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div><i><br>
// The following line produces
an error!!!<br>
GstGLMemory* glMemory =
GST_GL_MEMORY_CAST(gst_gl_base_memory_alloc(<br>
GST_GL_BASE_MEMORY_ALLOCATOR_CAST(allocator),
(GstGLAllocationParams*) params));<br>
<br>
gst_gl_allocation_params_free((GstGLAllocationParams
*)params);<br>
gst_object_unref(allocator);<br>
<br>
// Attach GstGLMemory object
into buffer, timestamp the buffer
and push it downstream<br>
GstBuffer* buffer =
gst_buffer_new();<br>
gst_buffer_append_memory(buffer,
GST_MEMORY_CAST(glMemory));<br>
<br>
// Put timestamps into buffer<br>
GST_BUFFER_PTS (buffer) =
timestamp;<br>
GST_BUFFER_DURATION (buffer) =
gst_util_uint64_scale_int (1,
GST_SECOND, 2);<br>
<br>
timestamp +=
GST_BUFFER_DURATION (buffer);<br>
GstFlowReturn ret;<br>
g_signal_emit_by_name(state->appsrc,
"push-buffer", buffer, &ret);<br>
<br>
if (ret != GST_FLOW_OK)<br>
{<br>
// Something wrong, stop
pushing.<br>
g_printerr("Something went
wrong: Pushing buffer into appsrc
is stopped.\n");<br>
return false;<br>
}<br>
<br>
return true;<br>
}<br>
</i></div>
<div><br>
</div>
<div>pushFrame produces the following
error:</div>
<div><font color="#ff0000">gst_gl_context_thread_add:
assertion
'context->priv->active_thread
== g_thread_self ()' failIed<br>
</font></div>
<div><font color="#ff0000"><br>
</font></div>
<div><font color="#000000">What am I
doing wrong or how can push
gpu texture to gstreamer?</font></div>
<div><font color="#000000"><br>
</font></div>
<div><font color="#000000">Thanks,</font></div>
<div><font color="#000000">Lusine</font></div>
<div><font color="#000000"><br>
</font></div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
</blockquote>
<br>
</body>
</html>