some notes for opengl plugin

[comicfans44]

recently I'm learning intergrating gstreamer  into my application (using
opengl plugin and context share),gstreamer and its opengl plugin is 
powerful,  
but the opengl plugin lacks some "key notes " in document which is 
important 
for context-share at some point. you can only find them in the demo/src.
without this, some hard-to-debug problem may block others who try to 
integrate gstreamer opengl. so I wrote them down here, wish these notes 
going 
into document and  help others to use opengl plugin easiler. 

feel free to correct .


following step shows the way I integrate gstreamer with opengl 
context-share 
into my app (which comes from qglwtextureshare demo in gst-plugins-bad
tests/example/gl/qt):

app create it's opengl context (ref as main context)->
gstreamer create share context with main context(ref as gstreamer's 
context)->
use fakesink hands-off signal to receive GstBuffer wrapped texture back ->
draw texture on main context


in this way, gstreamer process images in another thread opengl,you receive
the result texture ,and do anything you want in your main context.


note 1: you must unbind your main context until gstreamer's context create 
        complete,(that is, gst_gl_context_create function returned) I 
haven't 
        find too much about this restriction, but there's notes at   	
        qglrenderer.cpp:73
        

  
  
  for nvidia card (windows), not to do this will lead gstreamer's context
  create code failed everytime. (you may GetLastError=0xc00710dd 
  see 
http://stackoverflow.com/questions/21074447/undocumented-error-during-opengl-context-creation)
  I didn't test the incorrect code under Linux,but I guess result will be 
same

  for ATI card (Linux/windows), not do this will lead no error code 
reported
      when gstreamer creating context, but main context window just black!
      you may find sometimes after running some other correct context-share 
      app , the incorrect code magically worked !(I'm not joking) ,  you 
may
      think your app "worked" ,but if you restart PC , everything goes 
crazy.
  


for some toolkit(likes Qt) ,opengl context is created on GUI thread, and 
you
can only ensure this context binded to current thread in GUI draw callback.
you can share main context to gstreamer as follows:

1.1. get main context in the GUI draw callback (or any position you're sure 
     main context is binded to current thread.like 
Qt::QGLWidget::initializeGL
     use wglGetCurrentContext or something equal)

1.2. unbind it (QGLContext::doneCurrent or something equal), you can 
reference 
     qglrenderer.cpp:64
    
1.3  passing it to gstreamer-gl element's other-context property 
    (reference to qglrenderer.cpp:64).  you just need to do this for only 
    one opengl element. currently I set it to last opengl element in 
pipeline 
    and it works OK.

1.4 blocking wait in GUI draw callback until gstreamer's context share code 
complete,
    (or you must ensure GUI-toolkit will not rebind the previously unbinded 
    main context back or call opengl function after draw callback returned) 
.
    then you can rebind your main context back and do anything as usual.

since there's no "context-share complete" signal from gstreamer , so you
don't know which timepoint your blocking wait can wakeup, one choice is 
waiting until whole pipeline enter PAUSED state , like the qglwtextureshare 
demo. I'm not sure the state change workflow inside gstreamer .seems this 
only
works when src supports poll mode (like videotestsrc/filesrc ) ,
but if you're using appsrc in push mode , PAUSED state is not enough, 
you must wait pipeline enter PLAYING state (wait the gl element which you 
set 
other-context enter PLAYING state not helped ). if your data is not ready 
at the point, whole pipeline will not enter PLAYING state, you can pushing 
"fake data" once to appsrc at beginning (before waiting ) to avoid the UI 
block.

you should arrange app logic carefully , if you block waiting in UI thread 
and 
use main-loop based timer (g_idle_add or QTimer) in UI thread to trigger 
data push,  it may dead-lock (main loop blocked by waiting , but pipeline 
waiting main loop timer callback to push)



if you are using fakesink/appsink to receive data back, you can also wakeup
UI waiting when first buffer arrvied.


so here comes the point I think opengl plugin can be improved, add a
"context share complete" signal helps a lot, or maybe some other
ways to create share-context immediately, not depends on state change.


note 2. when using the GstBuffer wrapped texture in main context, you
   may flush the gstreamer context first. opengl may cache draw commands, 
   so the gstreamer's texture may not be complete before being used in main 
   context.

  for nvidia card, the gstreamer's  texture is always rendered completely
       in main context use(seems nv driver control this)

  for ATI card,without this, main context can render some
      gstreamer's textures correctly at first, but may fail after a while 
or 
      elements dynamically changed. you will find many random garbage 
      or previous frame drawed on main context.

  you may reference qglrenderer.cpp:122 to get code hint.(peek GLMemory 
  from result GstBuffer, add flush callback to GLMemory's context by 
  gst_gl_context_thread_add, it will execute flush callback in
  gstreamer's opengl thread, wait it complete and return)

  qglwtextureshare demo cached 3 GstBuffers before drawing on main context
    (pipeline.cpp:178),   reduce this number to 1 ,you will notice flick in 
    app without flushing . 
    
That's all, hopes it helped.