push opengl texture to gstreamer pipeline
Lusine Hayrapetyan
lusinehayrapetyan1992 at gmail.com
Tue Aug 17 14:13:58 UTC 2021
WRT *" retrieve the GStreamer created OpenGL context from one of the
downstream elements" *unfortunately it's not our use case.
WRT " *If you are not also providing your wrapped OpenGL in response to the
'need-context' message" *I'm providing it, this is how I handle
"need-context":
static void sync_bus_call (GstBus * bus, GstMessage * msg, gpointer * data)
{
APP_STATE_T *state = (APP_STATE_T *)data;
switch (GST_MESSAGE_TYPE (msg))
{
case GST_MESSAGE_NEED_CONTEXT:
{
g_print("-------------------------sync_bus_call got
GST_MESSAGE_NEED_CONTEXT--------------------\n");
const gchar *context_type;
gst_message_parse_context_type (msg, &context_type);
GstContext *context = NULL;
if (g_strcmp0 (context_type, GST_GL_DISPLAY_CONTEXT_TYPE) == 0)
{
g_print("-------------------------got
GST_GL_DISPLAY_CONTEXT_TYPE--------------------\n");
context = gst_context_new (GST_GL_DISPLAY_CONTEXT_TYPE,
TRUE);
// state->x_gst_display was created like
this: state->x_gst_display =
gst_gl_display_x11_new_with_display(state->xdisplay);
gst_context_set_gl_display(context,
GST_GL_DISPLAY(state->x_gst_display));
g_print("msg->src element is %s\n\n",
GST_ELEMENT_NAME(msg->src)); // This prints gldownload.
gst_element_set_context (GST_ELEMENT(msg->src), context);
}
else if (g_strcmp0 (context_type, "gst.gl.app_context") == 0)
{
g_print("-------------------------got
gst.gl.app_context--------------------\n");
GstContext *context = gst_context_new("gst.gl.app_context",
TRUE);
GstStructure *s = gst_context_writable_structure (context);
// state->gl_context is my wrapped context which was
created like this:
// state->gl_context = gst_gl_context_new_wrapped
(GST_GL_DISPLAY (state->x_gst_display),
// (guintptr)
state->context, GST_GL_PLATFORM_EGL, GST_GL_API_GLES2);
gst_structure_set (s, "context", GST_TYPE_GL_CONTEXT,
state->gl_context, NULL);
g_print("msg->src element is %s\n",
GST_ELEMENT_NAME(msg->src)); // This prints gldownload.
gst_element_set_context(GST_ELEMENT(msg->src), context);
}
break;
}
WRT: *"if you are creating another unwrapped OpenGL context with
gst_gl_display_create_context() then you have notify GStreamer about this
somehow"*
I use the unwrapped gl context with *gst_gl_display_add_context *and when
I'm creating GstGLMemory, is there other places I need to use it?
Best Regards,
Lusine
On Tue, Aug 17, 2021 at 4:52 PM Matthew Waters <ystreet00 at gmail.com> wrote:
> Ah,
>
> Ok. Now, if you are creating another unwrapped OpenGL context with
> gst_gl_display_create_context() then you have notify GStreamer about this
> somehow, or GStreamer will also create it's own. If you are not also
> providing your wrapped OpenGL in response to the 'need-context' message,
> then GStreamer OpenGL is not linked with your application OpenGL context.
>
> You should probably just retrieve the GStreamer created OpenGL context
> from one of the downstream elements (the 'context' property on most OpenGL
> elements) and use that.
>
> Cheers
> -Matt
>
> On 17/8/21 1:53 am, Lusine Hayrapetyan wrote:
>
> Got it! Thanks!
> Used *gst_gl_display_x11_new_with_display *instead of *gst_gl_display_egl_new_with_egl_display
> *and the warning about " *Wrong display type" *is fixed. I don't see any
> other warn/issue in the gl log but my initial issue still exists 😿 - I
> create GstGLMemory from texture_id and push the GstBuffer to appsrc and see
> this errors:
>
> 0:00:14.023472773 306156 0x55a56659d6a0 LOG glbasememory
> gstglbasememory.c:269:_map_data_gl: mapping mem 0x7f105c227d00 flags 20002
> 0:00:14.023553671 306156 0x55a56659d6a0 DEBUG gldebug
> gstgldebug.c:319:_gst_gl_debug_callback:<glcontextegl0> medium: GL other
> from API id:1, FBO incomplete: color attachment incomplete [0]
>
> (testegl1:306156): GStreamer-CRITICAL **: 15:39:40.709:
> gst_debug_log_valist: assertion 'category != NULL' failed
> 0:00:14.023741217 306156 0x55a56659d6a0 WARN glbasetexture
> gstglmemory.c:404:gst_gl_memory_read_pixels: Could not create framebuffer
> to read pixels for memory 0x7f10d08992b0
> 0:00:14.023776145 306156 0x55a56659d6a0 LOG glbasememory
> gstglbasememory.c:348:_unmap_data_gl: unmapping mem 0x7f105c227d00 flags
> 20002
> 0:00:14.023800095 306156 0x55a56659d6a0 LOG glbuffer
> gstglbuffer.c:305:_gl_buffer_unmap: unmapping 0x7f105c227d00 id 1 size
> 360000
> 0:00:14.024028993 306156 0x55a56659d6a0 LOG glbasememory
> gstglbasememory.c:269:_map_data_gl: mapping mem 0x7f10d08992b0 flags 10001
> 0:00:14.024074629 306156 0x55a56659d6a0 LOG glbasememory
> gstglbasememory.c:227:gst_gl_base_memory_alloc_data: 0x7f10d08992b0
> attempting allocation of data pointer of size 360007
> 0:00:14.024139538 306156 0x55a56659d6a0 DEBUG glbasememory
> gstglbasememory.c:236:gst_gl_base_memory_alloc_data: 0x7f10d08992b0
> allocated data pointer alloc 0x7f10c05fa010, data 0x7f10c05fa010
> 0:00:14.024169028 306156 0x55a56659d6a0 LOG glbasememory
> gstglbasememory.c:269:_map_data_gl: mapping mem 0x7f105c227d00 flags 20002
> 0:00:14.024195194 306156 0x55a56659d6a0 LOG glbasememory
> gstglbasememory.c:269:_map_data_gl: mapping mem 0x7f105c227d00 flags 20002
> 0:00:14.024217673 306156 0x55a56659d6a0 LOG glbasememory
> gstglbasememory.c:278:_map_data_gl: multiple map no 2 flags 20002 all flags
> 20002
> 0:00:14.024263516 306156 0x55a56659d6a0 DEBUG gldebug
> gstgldebug.c:319:_gst_gl_debug_callback:<glcontextegl0> medium: GL other
> from API id:1, FBO incomplete: color attachment incomplete [0]
>
> On Mon, Aug 16, 2021 at 6:55 PM Matthew Waters <ystreet00 at gmail.com>
> wrote:
>
>> If you're running on linux, there is a native display handle behind the
>> EGLDIsplay. That native display handle would be an X11 Display, or a
>> wayland wl_display (or maybe a mir display). OpenGL resources cannot be
>> shared unless this 'native' display type is the same between all OpenGL
>> uses. Passing an EGLDisplay is not sufficient and will not work.
>>
>> You need to pass the state->xdisplay using the appropriate
>> platform-specific gst_gl_display_x11_new_with_display(). You cannot use
>> the EGLDisplay version here.
>>
>> Cheers
>> -Matt
>>
>> On 17/8/21 12:40 am, Lusine Hayrapetyan wrote:
>>
>> Hi Mat,
>>
>> My display is *EGLDisplay *type:
>> /* get an EGL display connection */
>> EGLNativeDisplayType nativeDisplay = state->xdisplay;
>> *state->display* = eglGetDisplay (nativeDisplay);
>> assert (state->display != EGL_NO_DISPLAY);
>>
>> /* The I create an EGL rendering context with EGLDisplay */
>> state->context =
>> eglCreateContext (state->display, config,
>> EGL_NO_CONTEXT, context_attributes);
>>
>> /* Then I create GstGLDisplayEGL from EGLDisplay */
>> state->gst_display = gst_gl_display_egl_new_with_egl_display
>> (state->display);
>>
>> /* And create my wrapped context with GstGLDisplayEGL */
>> state->gl_context =
>> gst_gl_context_new_wrapped (GST_GL_DISPLAY (state->gst_display),
>> (guintptr) state->context, GST_GL_PLATFORM_EGL, GST_GL_API_GLES2);
>>
>> /* Now I want to create a shared context from my wrapped context and
>> again I pass GstGLDisplayEGL to it */
>> gst_gl_display_create_context(GST_GL_DISPLAY(state->gst_display),
>> state->gl_context, &state->newContext, &error) )
>>
>> And this function complains that it expects an x11 display instead of egl
>> display. But I don't understand how I can pass x11 display because my
>> context is created with egl display.
>> I'm not experienced in opengl es & egl, do I miss something?
>>
>> Regards,
>> Lusine
>>
>> On Fri, Aug 13, 2021 at 1:48 PM Matthew Waters <ystreet00 at gmail.com>
>> wrote:
>>
>>>
>>>
>>> On 13/8/21 5:59 pm, Lusine Hayrapetyan wrote:
>>>
>>> In gstglconfig.h I see the following:
>>>
>>> *#define GST_GL_HAVE_WINDOW_X11 1 *
>>>
>>> I've built the latest master on Ubuntu 20 and still see the same message
>>> about x11 window and there is a new FIXME message:
>>> 0:00:03.382786440 252231 0x55e6075596a0 INFO glwindow
>>> gstglwindow.c:295:gst_gl_window_new: creating a window, user choice:x11
>>> 0:00:03.382847275 252231 0x55e6075596a0 INFO glwindow
>>> gstglwindow_x11.c:136:gst_gl_window_x11_new: Wrong display type 32 for this
>>> window type 1
>>>
>>>
>>> This is the problematic line. The display type (EGL) as returned by
>>> gst_gl_display_get_type() does not match the window impl's expectations
>>> (X11). If you are running under X11 and your application is using X11 to
>>> connect to the display server, you need to pass an X11 GstGLDisplay.
>>> Passing an EGLDisplay is not sufficient and will not work. An appropriate
>>> EGLDIsplay will be created internally.
>>>
>>> Cheers
>>> -Matt
>>>
>>> 0:00:03.382877455 252231 0x55e6075596a0 WARN glwindow
>>> gstglwindow.c:340:gst_gl_window_new: Could not create window. user
>>> specified x11, creating dummy window
>>> 0:00:03.382998854 252231 0x55e6075596a0 DEBUG gldisplay
>>> gstgldisplay.c:694:gst_gl_display_create_window:<gldisplayegl0> Adding
>>> window <gldummywindow0> (0x55e607797c50) to internal list
>>> 0:00:03.383024753 252231 0x55e6075596a0 DEBUG glcontext
>>> gstglcontext.c:963:gst_gl_context_set_window:<glcontextegl0>
>>> window:<gldummywindow0>
>>> 0:00:03.383269595 252231 0x55e6077b5640 DEBUG glcontext
>>> gstglcontext.c:1223:gst_gl_context_create_thread:<glcontextegl0> Creating
>>> thread
>>> 0:00:03.383377039 252231 0x55e6077b5640 FIXME glcontext
>>> gstglcontext.c:2039:gst_gl_wrapped_context_get_config:<glwrappedcontext0>
>>> wrapped context could not retrieve config. The application may be missing a
>>> call to gst_gl_context_fill_info() or the specific platform implemention is
>>> not implemented for retrieving the config from a wrapped OpenGL context.
>>> 0:00:03.383437907 252231 0x55e6077b5640 INFO glcontext
>>> gstglcontext.c:1315:gst_gl_context_create_thread:<glcontextegl0> Attempting
>>> to create opengl context. user chosen api(s) (gles2), compiled api support
>>> (opengl opengl3 gles2) display api (gles2)
>>> 0:00:03.383463315 252231 0x55e6077b5640 DEBUG glcontext
>>> gstglcontext_egl.c:863:gst_gl_context_egl_create_context:<glcontextegl0>
>>> Creating EGL context
>>>
>>> Regards,
>>> Lusine
>>>
>>> On Thu, Aug 12, 2021 at 11:18 AM Matthew Waters <ystreet00 at gmail.com>
>>> wrote:
>>>
>>>> Yes, that is a critical error for X11. X11 would require access to an
>>>> X11 Display which must be provided by your application in order share
>>>> OpenGL resources. You may also not have X11 support built and you would
>>>> have to check your build configuration for that. gstglconfig.h would
>>>> contain information on what libgstgl-1.0 was built with.
>>>>
>>>> It looks like it's picking a raw EGLDisplay instead. This was
>>>> potentially an issue in previous versions of GStreamer that has been fixed
>>>> in master by
>>>> https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1154
>>>> and
>>>> https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1169
>>>> .
>>>>
>>>> Cheers
>>>> -Matt
>>>>
>>>> On 12/8/21 1:50 am, Lusine Hayrapetyan wrote:
>>>>
>>>> For limiting to a particular OpenGL variant I'm using the following
>>>> environment variables, is it the same?
>>>>
>>>>
>>>> * export GST_GL_WINDOW=x11 export GST_GL_PLATFORM=egl export
>>>> GST_GL_API=gles2*
>>>>
>>>> I've attached GST_DEBUG=gl*:7 output (gl_output.txt).
>>>> I see the following messages in the output:
>>>>
>>>>
>>>>
>>>>
>>>> *0:00:03.857084617 24438 0x55d59121d400 INFO glwindow
>>>> gstglwindow.c:278:gst_gl_window_new: creating a window, user choice:x11
>>>> 0:00:03.857117499 24438 0x55d59121d400 INFO glwindow
>>>> gstglwindow_x11.c:137:gst_gl_window_x11_new: Wrong display type 32 for this
>>>> window type 1 0:00:03.857144332 24438 0x55d59121d400 WARN
>>>> glwindow gstglwindow.c:324:gst_gl_window_new: Could not create window.
>>>> user specified x11, creating dummy window 0:00:03.857277761 24438
>>>> 0x55d59121d400 DEBUG gldisplay
>>>> gstgldisplay.c:579:gst_gl_display_create_window:<gldisplayegl0> Adding
>>>> window <gldummywindow0> (0x55d5913809d0) to internal list 0:00:03.857311159
>>>> 24438 0x55d59121d400 DEBUG glcontext
>>>> gstglcontext.c:948:gst_gl_context_set_window:<glcontextegl0>
>>>> window:<gldummywindow0>*
>>>>
>>>> Is this a critical issue? Can this cause a texture reading issue?
>>>>
>>>> On Tue, Aug 10, 2021 at 10:21 AM Matthew Waters <ystreet00 at gmail.com>
>>>> wrote:
>>>>
>>>>> Ah, if you want to limit to a particular OpenGL variant, you need to
>>>>> also call gst_gl_display_filter_gl_api() with that variant.
>>>>>
>>>>> GST_DEBUG=gl*:7 will output a whole bunch of information about what is
>>>>> generated and tried in all of this respect.
>>>>>
>>>>> Cheers
>>>>> -Matt
>>>>>
>>>>> On 9/8/21 12:58 am, Lusine Hayrapetyan wrote:
>>>>>
>>>>> Hi Matthew,
>>>>> Seems the second approach fits to my use case- I need to push textures
>>>>> which are created in opengles & egl( it means not in gstreamer context) to
>>>>> gstreamer.
>>>>> I've changed my code to use gst_gl_display_create_context & gst_gl_display_add_context
>>>>> but still can't read textures in gstreamer, gstreamer prudeces the
>>>>> following errors after I push buffer to appsrc:
>>>>> (testegl1:24602): GStreamer-CRITICAL **: 13:02:55.568:
>>>>> gst_debug_log_valist: assertion 'category != NULL' failed
>>>>> 0:00:14.039444188 24602 0x5611bc356d90 WARN glbasetexture
>>>>> gstglmemory.c:401:gst_gl_memory_read_pixels: Could not create framebuffer
>>>>> to read pixels for memory 0x7f8cf0017ac0
>>>>> 0:00:14.039480909 24602 0x5611bc356d90 WARN glbasememory
>>>>> gstglbasememory.c:585:gst_gl_base_memory_memcpy: could not read map source
>>>>> memory 0x7f8cf0017ac0
>>>>> 0:00:14.039505741 24602 0x5611bc356d90 WARN glmemory
>>>>> gstglmemorypbo.c:592:_gl_mem_copy: Could not copy GL Memory
>>>>> 0:00:14.039779268 24602 0x5611bc4f14f0 ERROR videometa
>>>>> gstvideometa.c:247:default_map: cannot map memory range 0-1
>>>>> 0:00:14.039846056 24602 0x5611bc4f14f0 ERROR default
>>>>> video-frame.c:168:gst_video_frame_map_id: failed to map video frame plane 0
>>>>> 0:00:14.039891314 24602 0x5611bc4f14f0 WARN videofilter
>>>>> gstvideofilter.c:297:gst_video_filter_transform:<videoconvert0> warning:
>>>>> invalid video buffer received
>>>>>
>>>>>
>>>>> This is how I implemented it, did I misunderstand something?
>>>>>
>>>>> I have rendering thread where I initialize opengles context and create
>>>>> wrapped and the new contexts.
>>>>> //
>>>>> // Description: Sets the display, OpenGL|ES context and screen stuff
>>>>> // Created GstGLContext s - wrapped (gst_gl_context_new_wrapped) and
>>>>> new context(gst_gl_display_create_context)
>>>>> //
>>>>> static void
>>>>> init_ogl (APP_STATE_T * state)
>>>>> {
>>>>> ...
>>>>> /* get an EGL display connection */
>>>>> state->display = eglGetDisplay (EGL_DEFAULT_DISPLAY);
>>>>> assert (state->display != EGL_NO_DISPLAY);
>>>>> /* initialize the EGL display connection */
>>>>> result = eglInitialize (state->display, NULL, NULL);
>>>>> assert (EGL_FALSE != result);
>>>>>
>>>>> /* create an EGL rendering context */
>>>>> state->context =
>>>>> eglCreateContext (state->display, config,
>>>>> EGL_NO_CONTEXT, context_attributes);
>>>>> assert (state->context != EGL_NO_CONTEXT);
>>>>> //
>>>>> // Initialize GStreamer related resources.
>>>>> //
>>>>> state->gst_display = gst_gl_display_egl_new_with_egl_display
>>>>> (state->display);
>>>>> state->gl_context =
>>>>> gst_gl_context_new_wrapped (GST_GL_DISPLAY (state->gst_display),
>>>>> (guintptr) state->context, GST_GL_PLATFORM_EGL,
>>>>> GST_GL_API_GLES2);
>>>>>
>>>>> GError *error = NULL;
>>>>> if (
>>>>> !gst_gl_display_create_context(GST_GL_DISPLAY(state->gst_display),
>>>>> state->gl_context, &state->newContext, &error) )
>>>>> g_print("Failed to create new context\n");
>>>>>
>>>>> if ( !gst_gl_display_add_context(GST_GL_DISPLAY(state->gst_display),
>>>>> state->newContext))
>>>>> g_print("Failed to add new context to display\n");
>>>>>
>>>>> } // init_ogl end.
>>>>>
>>>>>
>>>>> static void
>>>>> sync_bus_call (GstBus * bus, GstMessage * msg, gpointer * data)
>>>>> {
>>>>>
>>>>> APP_STATE_T *state = (APP_STATE_T *)data;
>>>>> switch (GST_MESSAGE_TYPE (msg))
>>>>> {
>>>>> case GST_MESSAGE_NEED_CONTEXT:
>>>>> {
>>>>> const gchar *context_type;
>>>>> gst_message_parse_context_type (msg, &context_type);
>>>>>
>>>>> GstContext *context = NULL;
>>>>> if (g_strcmp0 (context_type, GST_GL_DISPLAY_CONTEXT_TYPE)
>>>>> == 0)
>>>>> {
>>>>> GstGLDisplay * gl_display =
>>>>> GST_GL_DISPLAY(state->gst_display);
>>>>> context = gst_context_new (GST_GL_DISPLAY_CONTEXT_TYPE,
>>>>> TRUE);
>>>>> gst_context_set_gl_display(context, gl_display);
>>>>> gst_element_set_context (GST_ELEMENT(msg->src),
>>>>> context);
>>>>> }
>>>>> else if (g_strcmp0 (context_type, "gst.gl.app_context") ==
>>>>> 0)
>>>>> {
>>>>> GstContext *context =
>>>>> gst_context_new("gst.gl.app_context", TRUE);
>>>>> GstStructure *s = gst_context_writable_structure
>>>>> (context);
>>>>> gst_structure_set (s, "context", GST_TYPE_GL_CONTEXT,
>>>>> state->gl_context, NULL);
>>>>> gst_element_set_context(GST_ELEMENT(msg->src),
>>>>> context);
>>>>> }
>>>>> break;
>>>>> }
>>>>> default:
>>>>> break;
>>>>> }
>>>>> } // sync_bus_call end
>>>>>
>>>>>
>>>>> I use need-data callback to create a buffer from texture_id and and
>>>>> push it in the appsrc:
>>>>> *g_signal_connect (state->appsrc, "need-data", G_CALLBACK (*
>>>>> pushFrame *), state);*
>>>>>
>>>>> static bool pushFrame(..., APP_STATE_T * state)
>>>>> {
>>>>> // Wrap the texture into GstGLMemory
>>>>> GstVideoInfo vinfo;
>>>>> gst_video_info_set_format(&vinfo, GST_VIDEO_FORMAT_RGBA,
>>>>> state->screen_width, state->screen_height);
>>>>> // Use state->newContext for allocator.
>>>>> GstAllocator* allocator =
>>>>> GST_ALLOCATOR(gst_gl_memory_allocator_get_default(state->newContext));
>>>>> GstGLVideoAllocationParams* params =
>>>>> gst_gl_video_allocation_params_new_wrapped_texture(
>>>>> state->newContext, NULL, &vinfo, 0, NULL,
>>>>> GST_GL_TEXTURE_TARGET_2D, GST_GL_RGBA, state->tex,
>>>>> NULL, 0);
>>>>>
>>>>> GstGLMemory* glMemory =
>>>>> GST_GL_MEMORY_CAST(gst_gl_base_memory_alloc(
>>>>> GST_GL_BASE_MEMORY_ALLOCATOR_CAST(allocator),
>>>>> (GstGLAllocationParams*) params));
>>>>>
>>>>> gst_gl_allocation_params_free((GstGLAllocationParams *)params);
>>>>> gst_object_unref(allocator);
>>>>>
>>>>> // Attach GstGLMemory object into buffer, timestamp the buffer and
>>>>> push it downstream
>>>>> GstBuffer* buffer = gst_buffer_new();
>>>>> gst_buffer_append_memory(buffer, GST_MEMORY_CAST(glMemory));
>>>>>
>>>>> GstVideoMeta * vmeta = gst_buffer_add_video_meta(buffer,
>>>>> GST_VIDEO_FRAME_FLAG_NONE, GST_VIDEO_FORMAT_RGBA, state->screen_width,
>>>>> state->screen_height);
>>>>>
>>>>> // Put timestamps into buffer
>>>>> GST_BUFFER_PTS (buffer) = timestamp;
>>>>> GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int (1,
>>>>> GST_SECOND, 2);
>>>>> timestamp += GST_BUFFER_DURATION (buffer);
>>>>>
>>>>> GstFlowReturn ret;
>>>>> g_signal_emit_by_name(state->appsrc, "push-buffer", buffer, &ret);
>>>>>
>>>>> if (ret != GST_FLOW_OK)
>>>>> {
>>>>> // Something wrong, stop pushing.
>>>>> g_printerr("Something went wrong: Pushing buffer into appsrc
>>>>> is stopped.\n");
>>>>> return false;
>>>>> }
>>>>>
>>>>> return true;
>>>>> } // pushFrame end
>>>>>
>>>>> Regards,
>>>>> Lusine
>>>>>
>>>>> On Fri, Aug 6, 2021 at 8:59 AM Matthew Waters <ystreet00 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> On 6/8/21 1:12 am, Lusine Hayrapetyan wrote:
>>>>>>
>>>>>> Dear Matt,
>>>>>> Thank you very much for your response. It helped me to understand
>>>>>> that using 'wrapped' OpenGL context is a wrong direction to go :)
>>>>>>
>>>>>> *WRT Suggestion 1:*
>>>>>> Do I understand correctly that I need to get local context from
>>>>>> gstreamer and pass it to opengl rendering thread? (I have a rendering
>>>>>> thread where I set OpenGL|ES context and screen stuff).
>>>>>> GstGLContext* mContext = nullptr;
>>>>>> g_object_get(state->gldownload, "context", &mContext, NULL);
>>>>>> guintptr handle;
>>>>>> handle = gst_gl_context_get_gl_context(mContext); // is this correct?
>>>>>> state->context = (EGLContext)handle; // state->context is EGLContext
>>>>>> type;
>>>>>> And then use state->context in OpenGL|ES?
>>>>>> Do I need to get and pass window and display from gstreamer to my
>>>>>> rendering thread as well?
>>>>>> Although my use scenario is different from this one - I need to pass
>>>>>> context from OpenGL to Gstreamer.
>>>>>>
>>>>>>
>>>>>> You only need to retrieve or create a non-wrapped GstGLContext and
>>>>>> use that for creating your textures that you are pushing into GStreamer.
>>>>>> You don't need to use GStreamer's provided OpenGL context for anything
>>>>>> else. Everything else in your sample remains the same. You may need to
>>>>>> add a GstGLSyncMeta on your buffers you are pushing into GStreamer to
>>>>>> provide the necessary synchronisation guarantees between the shared OpenGL
>>>>>> contexts (application and GStreamer). On some platforms the window handle
>>>>>> type and format may be important however in general on linux (X11/wayland)
>>>>>> it doesn't really matter.
>>>>>>
>>>>>> You must not attempt to use GStreamer's OpenGL context as-is (using
>>>>>> e.g. eglMakeCurrent() or anything of the like) from outside the GStreamer
>>>>>> OpenGL context thread (as provided by the gst_gl_context_thread_add() API).
>>>>>>
>>>>>> *WRT Suggestion 2:*
>>>>>> gst_gl_display_create_context accepts *other_context *argument,
>>>>>> should the *other_context *be the 'wrapped' context?
>>>>>>
>>>>>>
>>>>>> Yes. other_context is the GstGLContext that will be shared with the
>>>>>> newly created GstGLContext.
>>>>>>
>>>>>> Side note, GStreamer cannot use any application-provided OpenGL
>>>>>> context as-is due to the overhead of dealing with all the OpenGL state that
>>>>>> may be changed behind GStreamer's back. This is why the OpenGL context
>>>>>> sharing dance is required.
>>>>>>
>>>>>> Cheers
>>>>>> -Matt
>>>>>>
>>>>>> Best Regards,
>>>>>> Lusine
>>>>>>
>>>>>> On Thu, Aug 5, 2021 at 12:39 PM Matthew Waters <ystreet00 at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> So, I think you almost have the correct sequence.
>>>>>>>
>>>>>>> Response inline.
>>>>>>>
>>>>>>> On 5/8/21 1:04 am, Lusine Hayrapetyan via gstreamer-devel wrote:
>>>>>>>
>>>>>>> Hi Folks,
>>>>>>> I'm struggling with the following issue and can't understand what
>>>>>>> I'm doing wrong.
>>>>>>> I need to pass opengl texture to the gstreamer pipeline.
>>>>>>> I have a rendering thread where I create opengl texture, the
>>>>>>> following objects created in this thread:
>>>>>>> *EGLDisplay display;*
>>>>>>> *EGLContext context;*
>>>>>>>
>>>>>>> I create gstreamer pipeline in the main thread and as described in
>>>>>>> the following article sharing an X11 display and GstGLContext with the bus
>>>>>>> callback.
>>>>>>>
>>>>>>> http://ystreet00.blogspot.com/2015/09/gstreamer-16-and-opengl-contexts.html
>>>>>>>
>>>>>>> GstGLDisplayEGL and GstGLContext are created in this way:
>>>>>>>
>>>>>>> *GstGLDisplayEGL* gst_display =
>>>>>>> gst_gl_display_egl_new_with_egl_display (display); *
>>>>>>>
>>>>>>>
>>>>>>> *GstGLContext *gl_context = gst_gl_context_new_wrapped
>>>>>>> (GST_GL_DISPLAY (gst_display), (guintptr) context,
>>>>>>> GST_GL_PLATFORM_EGL, GST_GL_API_GLES2);*
>>>>>>>
>>>>>>> The first element of my pipeline is appsrc:
>>>>>>> *appsrc stream-type=0 emit-signals=1 format=3
>>>>>>> caps=video/x-raw(memory:GLMemory), width=300, height=300,
>>>>>>> framerate=(fraction)20/1, format=(string)RGBA ! gldownload ! ...*
>>>>>>>
>>>>>>> I use need-data callback to create a buffer from texture_id and and
>>>>>>> push it in the appsrc:
>>>>>>> *g_signal_connect (state->appsrc, "need-data", G_CALLBACK (*
>>>>>>> pushFrame *), state);*
>>>>>>>
>>>>>>> *bool pushFrame()*
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *{ // Wrap the texture into GstGLMemory GstVideoInfo vinfo;
>>>>>>> gst_video_info_set_format(&vinfo, GST_VIDEO_FORMAT_RGBA, 300, 300);
>>>>>>> GstAllocator* allocator =
>>>>>>> GST_ALLOCATOR(gst_gl_memory_allocator_get_default(gl_context));
>>>>>>> GstGLVideoAllocationParams* params =
>>>>>>> gst_gl_video_allocation_params_new_wrapped_texture(
>>>>>>> state->gl_context, NULL, &vinfo, 0, NULL, GST_GL_TEXTURE_TARGET_2D,
>>>>>>> GST_GL_RGBA, * texture_id
>>>>>>>
>>>>>>> *, NULL, 0); *
>>>>>>>
>>>>>>>
>>>>>>> The use of state->gl_context is probably your OpenGL context that
>>>>>>> has been wrapped from the application. This 'wrapped' OpenGL context has
>>>>>>> some limitations, one being that GStreamer cannot actually do a complete
>>>>>>> gst_gl_context_thread_add where the request is marshalled to an
>>>>>>> OpenGL-specific thread. This is what that critical is complaining about
>>>>>>> effectively.
>>>>>>>
>>>>>>> To do this properly, you would need to do one of two things:
>>>>>>> 1. Retrieve the OpenGL context from the downstream gldownload
>>>>>>> element using either the 'context' property or using an appropriate
>>>>>>> GST_CONTEXT QUERY or the helper gst_gl_query_local_gl_context().
>>>>>>> 2. Create your own GStreamer OpenGL context and add it to the
>>>>>>> GstGLDisplay using something like:
>>>>>>> https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/gst-libs/gst/gl/gstglbasefilter.c#L550-565
>>>>>>> .
>>>>>>>
>>>>>>> Cheers
>>>>>>> -Matt
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> * // The following line produces an error!!! GstGLMemory*
>>>>>>> glMemory = GST_GL_MEMORY_CAST(gst_gl_base_memory_alloc(
>>>>>>> GST_GL_BASE_MEMORY_ALLOCATOR_CAST(allocator), (GstGLAllocationParams*)
>>>>>>> params)); gst_gl_allocation_params_free((GstGLAllocationParams
>>>>>>> *)params); gst_object_unref(allocator); // Attach GstGLMemory
>>>>>>> object into buffer, timestamp the buffer and push it downstream
>>>>>>> GstBuffer* buffer = gst_buffer_new(); gst_buffer_append_memory(buffer,
>>>>>>> GST_MEMORY_CAST(glMemory)); // Put timestamps into buffer
>>>>>>> GST_BUFFER_PTS (buffer) = timestamp; GST_BUFFER_DURATION (buffer) =
>>>>>>> gst_util_uint64_scale_int (1, GST_SECOND, 2); timestamp +=
>>>>>>> GST_BUFFER_DURATION (buffer); GstFlowReturn ret;
>>>>>>> g_signal_emit_by_name(state->appsrc, "push-buffer", buffer, &ret); if
>>>>>>> (ret != GST_FLOW_OK) { // Something wrong, stop pushing.
>>>>>>> g_printerr("Something went wrong: Pushing buffer into appsrc is
>>>>>>> stopped.\n"); return false; } return true; } *
>>>>>>>
>>>>>>> pushFrame produces the following error:
>>>>>>> gst_gl_context_thread_add: assertion 'context->priv->active_thread
>>>>>>> == g_thread_self ()' failIed
>>>>>>>
>>>>>>> What am I doing wrong or how can push gpu texture to gstreamer?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Lusine
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210817/fdb62a03/attachment-0001.htm>
More information about the gstreamer-devel
mailing list