Memory allocation error when trying to map a video frame

[Matthew Waters]

On 07/07/17 01:44, jeremi.wojcicki wrote:
> Dear all,
>
> I am developing an application for Android (using ndk, my code is inspired
> by tutorial-3) in which I want to redirect video stream to an OpenGL (ES
> 3.0) texture, that later on will be used in own rendering thread. Following
> some very useful threads [1] on this forum I've come up with this pipeline:
>
> videotestsrc !
> video/x-raw,format=RGB,width=1280,height=720,framerate=30/1,pixel-aspect-ratio=1/1
> ! glupload ! appsink
>
> I have managed to successfully share my renderers OpenGL context with
> gstreamer in a response to a bus call [2] GST_MESSAGE_NEED_CONTEXT:
>
>             // get GStreamer display from the native OpenGL EGL vars
>             gst_gl_display =
> gst_gl_display_egl_new_with_egl_display(gl_display);
>             gst_gl_context = gst_gl_context_new_wrapped(gst_gl_display,
> *gl_context,
>                                                         GST_GL_PLATFORM_EGL,
> GST_GL_API_GLES2);
>
>             // set the context
>             context = gst_context_new("gst.gl.app_context", TRUE);
>             s = gst_context_writable_structure(context);
>             gst_structure_set(s, "context", GST_GL_TYPE_CONTEXT,
> gst_gl_context, NULL);
>
>             gst_element_set_context(GST_ELEMENT (message->src), context);
>
> Callback is called two times during run-time, so I expect it is for the
> glupload and appsink elements, which seems logical to me. I set appsink
> "new-sample" callback and there I get my sample, extract buffer. I also get
> the current caps from the appsink sinkpad, to seem whether the caps were
> properly negotiated and I receive this:
> appsink0 caps: video/x-raw(memory:GLMemory), format=(string)RGB,
> width=(int)1280, height=(int)720, framerate=(fraction)30/1,
> multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1,
> interlace-mode=(string)progressive, texture-target=(string)2D
>
> Sample seem to be in the GL memory and I wanted! However when I then try to
> map the frame [3] with:
>
> gst_video_frame_map(&frames[robin], &info, buffer, (GstMapFlags)
> (GST_MAP_READ | (GST_MAP_FLAG_LAST << 1)) )
>
> I receive a follwing error:
>
>            gstgldebug.c:303:_gst_gl_debug_callback:<glcontextegl0> high: GL
> error from API id:150, Error:glGetQueryObjectui64vEXT::invalid query object
>             gstgldebug.c:303:_gst_gl_debug_callback:<glcontextegl0> high: GL
> error from API id:1, Error:glBeginQueryEXT::failed to allocate CPU memory
>             gstgldebug.c:303:_gst_gl_debug_callback:<glcontextegl0> high: GL
> error from API id:148, Error:glEndQueryEXT::query name is 0
>
> Even tough this error looks quite horrible the software does not crash...
> Which is good on one side, but I surely I do not want to leave it this way.
> The texture id I receive from the appsink is valid and OpenGL renders it
> well (I pass it the my renderer object which is global, but I use
> pthread_mutex to avoid any unexpected behaviours). Here is the full code of
> my appsink callback:

This is most likely a bug in libgstgl not creating query objects
correctly on your platform.

Can you file a bug about this with at least a GST_DEBUG=gl*:6 log and
the hardware you're running this on?

> https://hastebin.com/lozujofuda.hs <https://hastebin.com/lozujofuda.hs>  
>
> One more problem that I struggle with, however I do not know whether it is
> related to this error, is that sometimes when I run the app (~25% of times)
> actually the texture (from test source) does not show up. I keep getting
> frames and valid texture IDs but the texture after rendering is just black.
>
> I would be thankful for for some guidance with this issue, because it is
> quite a "niche" problem and I do not seem to finding anything on google.

A couple of things about this approach.

1. You need to be careful with references.  The texture id is only
guaranteed to be valid while there is a mapping open on it.  I see
you're passing the texture handle to another function.  If that function
stores the texture and it is referenced later, then the output is
undefined as you don't actually hold a reference to the texture.
2. When using multiple OpenGL contexts, one needs to ensure that the
synchronisation is correct between them.  There is a meta on most
video/x-raw(memory:GLMemory) buffers called GstGLSyncMeta that performs
this.  The flow is set sync point in one GL context, wait in the other. 
The other requirement is that after you've synchronized the GL contexts,
you need to rebind the texture with glBindTexture for the updates to be
propogated correctly to the new GL context.

Hope that helps.
-Matt

> Refs
> [1] 
> http://gstreamer-devel.966125.n4.nabble.com/OpenGL-Texture-via-GstGLUpload-So-close-Proper-Post-td4670295.html
> <http://gstreamer-devel.966125.n4.nabble.com/OpenGL-Texture-via-GstGLUpload-So-close-Proper-Post-td4670295.html>  
> [2] 
> http://ystreet00.blogspot.it/2015/09/gstreamer-16-and-opengl-contexts.html
> <http://ystreet00.blogspot.it/2015/09/gstreamer-16-and-opengl-contexts.html>  
> [3] 
> https://lists.freedesktop.org/archives/gstreamer-devel/2015-March/052126.html
> <https://lists.freedesktop.org/archives/gstreamer-devel/2015-March/052126.html>  

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 516 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20170707/10ca7ce7/attachment.sig>