Capturing an OpenGL window and re displaying it using GStreamer Appsrc

[Matthew Waters]

Hi,

On 26/03/14 13:36, fahad wrote:
> AppSrcTest.cpp
> <http://gstreamer-devel.966125.n4.nabble.com/file/n4666109/AppSrcTest.cpp>
>
> I started by using the appsrc example and extended it to feed raw pixel
> data. I am attempting to see what kind of delay I will come across during
> this process. i am having two issues at this point.
>
> 1-Capturing the window using glReadPixels() kind of works but end result in
> gstreamer output windows is not the same, its upside down and all over the
> place. I think i need to come up with a better caps filter or further modify
> the captured pixel before feeding it to appsrc, but at this point I cannot
> figure out what to do

OpenGL's coordinate system is 'generally' upside down compared to 
gstreamer and other image libraries.
That is, (0,0) in OpenGL is the bottom left of the screen whereas (0,0) 
is normally the top left.  I say generally because you can change it 
somewhat.

> 2-There is a significant delay between the OGL and Gstreamer window, is
> there a possibility to improve this delay some how. My end goal would be to
> try and transport the captured window by encoding it to mpeg2 and broadcast
> over UDP. Just from this simple result it would seem that the delay is way
> too much to even attempt that.

Readback is slow.  Nothing you can really do about it except attempt to 
pipeline the processes.  I would suggest placing a queue after the 
appsrc and before the videoconvert.  The other option is to seperate the 
gl drawing code from the gl readpixels code and run them in two separate 
threads but that involves threading and context sharing and stuff :).

Noticed a couple of things in your code.  You set the caps format field 
to RGB16 but seem to be asking for RGB24 data from OpenGL. Also, 
GST_TYPE_FRACTION, 30, 60 is equivalent to 1 frame every two seconds :)

> any pointers/suggestions are appreciated.

There's also gst-plugins-gl for gstreamer 1.2 
http://cgit.freedesktop.org/gstreamer/gst-plugins-gl (I think it still 
compiles there) which has been moved into gst-plugins-bad for 1.4.

Take a look at gltestsrc for an example of a source producing OpenGL 
textures 
http://cgit.freedesktop.org/gstreamer/gst-plugins-gl/tree/gst/gl/gstgltestsrc.c

> Running on Windows with FreeGlut and GStreamer 1.0 (1.2.1)