gst-gl-plugin: texture share problem and customized GstGLContext/Window

[Matthew Waters]

Hi,

On 03/03/14 12:03, comicfans44 wrote:
> Hello everyone ,  I'm trying to use Qt GraphicsScene ( based on OpenGL
> painting,draw overlay items)  and Gst-plugin-opengl(draw
> background)together,after some try I came to some problem (linux + X11):
>
>   first I try to share GL context between Qt and Gstreamer thread
>   (option A), walked some code I found  share context create also needs
>   x11 Display*  so I did as following:
>
> 1 :  in Qt GL callback,  use glXGetCurrentContext get current GL
> context ,  I don't know how to get  x11 Display*  from Qt so I
> use XOpenDisplay(getenv("DISPLAY")) to open One

Another option is to create your display normally and then pass that 
into gst-gl using the GstContext query.

> 2:   make up GstGLWindowX11/GstGLContextGLX , fill the GL context and
> *Display
>
> 3:  pass the GstGLContextGLX*  to a   global variable ,  hack
> gst_gl_context_glx_create_context  to use this variable (which will run
> the share context code path) .    I didn't know how to do this step
> through gst-gl public API , since  GstGLImageSink always call
> gst_gl_context_create  with shared context=0 . GstGLFilter has
> other-context property but I did not always have a GstGLFilter all the
> time.

Hmm, it should be possible to add that property to glimagesink, the 
workaround, as you suggested, is to use some GstGLFilter subclass 
(glcolorscale or gleffects with effect property set to 0).  Also, you 
should use gst_gl_context_new_wrapped to wrap an existing GL context.

> 4:  init gstreamer pipeline  in Qt GL callback thread (  if
> I init gstreamer pipeline in other thread, context share will fail).
> since Qt already has glib mainloop, gstreamer  is started .
>
> 5:  use glimagesink -> clientDrawCallback to receive the result
> texture from gstreamer , then pass it to Qt OpenGL thread , since this
> texture id is swapped rapidly , I  blocked in clientDrawCallback ,wait
> until Qt GL draw complete.   in Qt GL draw callback,  I can successfully
> use this texture ,draw as background , then call Qt's painting function
> to draw Qt's items. (after all,  wake up clientDrawCallback thread.)
>
> everything draws OK but app locks randomly.  I  think this is not
> deadlock caused by my thread logic , if using gdb  attach to locked
> app, no thread is blocking by my thread code (mutex or waitcondition),
> but there's always thread blocked in fglrx_dri.so ( I'm using catalyst )
> this is the thread backtrace like:
>
> Thread 3 (Thread 0xb13e0b40 (LWP 6280)):
> #0  0xb7740424 in __kernel_vsyscall ()
> #1  0xb5f7fcc5 in sem_wait@@GLIBC_2.1 ()
> from /lib/i386-linux-gnu/libpthread.so.0
> #2  0xb3c80969 in ?? () from /usr/lib/fglrx/dri/fglrx_dri.so
> #3  0x08c5b220 in ?? ()
> Backtrace stopped: previous frame identical to this frame (corrupt
> stack?)
>
>
> Thread 2 (Thread 0xb05ffb40 (LWP 6281)):
> #0  0xb7740424 in __kernel_vsyscall ()
> #1  0xb5f7d96b in pthread_cond_wait@@GLIBC_2.3.2 ()
>     from /lib/i386-linux-gnu/libpthread.so.0
> #2  0xb61694bc in pthread_cond_wait ()
> from /lib/i386-linux-gnu/libc.so.6
> #3  0xb64b17c6 in g_cond_wait ()
> from /lib/i386-linux-gnu/libglib-2.0.so.0
> #4  0xb63be7c4 in gst_gl_window_default_send_message (window=0x8c370e0,
>      callback=0xb63a36b0 <_gst_gl_context_thread_run_generic>,
> data=0xb05fec54)
>      at ../../../../gst-libs/gst/gl/gstglwindow.c:343
> #5  0xb63be66c in gst_gl_window_send_message (window=0x8c370e0,
>      callback=0xb63a36b0 <_gst_gl_context_thread_run_generic>,
> data=0xb05fec54)
>      at ../../../../gst-libs/gst/gl/gstglwindow.c:370
> #6  0xb63a5180 in gst_gl_context_thread_add (context=0x8c358b0,
> ---Type <return> to continue, or q <return> to quit---
>      func=0xb63baf00 <_do_upload>, data=0xafc082b0)
>      at ../../../../gst-libs/gst/gl/gstglcontext.c:867
>
> ......(following backtrace is from gstreamer itself and I omit it)
>
>
> Thread 1 (Thread 0xb5d5bac0 (LWP 6279)):
> #0  0xb7740424 in __kernel_vsyscall ()
> #1  0xb5f7fcc5 in sem_wait@@GLIBC_2.1 ()
> from /lib/i386-linux-gnu/libpthread.so.0
> #2  0xb3c80969 in ?? () from /usr/lib/fglrx/dri/fglrx_dri.so
> Backtrace stopped: previous frame inner to this frame (corrupt stack?)
>
>
> plus , I found that if I draw the  result texture in both
> thread , more random locks.(I wait Qt GL draw completed then draw it in
> clientDrawCallback,so texture is not used at the same time. at least
> at my opengl code level)
>
> so , I wonder if this is a problem in my misused code , or a bug in
> driver?

Sounds like a driver bug to me.  Unfortunately I do not have any radeon 
GPU's to test with.  You could alternatively try with the mesa stack 
(which AMD is contributing to)

> then I came to another approach, using glimagesink ,but write
> a custom GstGLContext/GstGLWindow (option B),  deliver all opengl
> callback to Qt GL thread, with some hack in gst-gl ,finally got it
> worked with no random lock, but this is a temp hack .
>
>
> I also tried to use appsink to receive GstBuffer  from opengl
> element(option C), with some code copied from glimagesink(including
> GstGLUpload, with gst_gl_upload_perform_with_buffer/
> gst_gl_upload_release_buffer pair), I can receive gl texture ,but the
> texture id always increases in every buffer receive (seems texture
> never freed ), instead of two id swapping as glimagesink, I don't know
> which part of code does the trick  ?

Bufferpools allow the reuse of GstBuffer's.  And GstBuffer's contain 
GstMemory's and there's a GstGLMemory which holds a GL texture. Although 
that *should* already be used.

> can somebody provide some suggestions? if sharing texture between
> thread (option A) is not stable, should we provide a easier way to use
> custom GstGLContext/GstGLWindow (option B) ?  or how to use appsink
> (option C)to receive buffer and correctly release it ? Thanks
>
>
>

It would be stupendous if you could file a bug here 
https://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer against 
gst-plugins-gl with the example program/s.