[gst-devel] video sink to update OpenGL texture?

[Andres Colubri]

Hi, I have been following this discussion with interest and I just 
wanted to add a couple of comments/questions.

I'm currently combining opengl and gstreamer within my application. 
Right now, the gst-originated video buffers are pushed out of the 
pipeline with a handoff signal, and then uploaded to an opengl texture 
for further gpu-accelerated processing and final display on the 
application window.

But it would be quite great if I can just get the gl texture id from the 
gst-plugins, for instance glupload, and then directly use the texture 
inside my application.

Since my app is cross-platform (linux, win, osx), the composite 
redirection approach doesn't seem viable. The XOverlay interface won't 
work either, because I need to do some gpgpu post-processing before the 
actual image rendering.

 From what I read, one alternative (perhaps the only one under this type 
of scenarios) would be to share the opengl context between the app and 
the gst gl-plugin. Basically, the app would create the opengl context 
and then pass it to the plugin when initializing the gst pipeline.

Is this possible in the current implementation of gst-plugins? If not, 
wouldn't be an useful (additional) method for setting up gl-accelerated 
pipelines?

Thanks,
Andres

Florent wrote:
> Hi,
>
>   
>> Thanks Jan, I installed gst-plugins-gl from git and did some first
>> experiments. This seems to be what I'm looking for. Is it correct that
>> I would use 'glimagesink' and do the drawing using the
>> client-draw-callback? I'm not experienced with OpenGL, is it safe to
>> just redraw the part of the scene that shows the video in the
>> callback?
>>     
>
> If i'm not mistaken glimagesink isn't what you are looking for, except
> if your opengl application supports composite redirection (when
> glimagesink's textures are redirected inside your own opengl app): the
> glimagesink will be a dedicated, separate window and gl context. If
> you need to display gst-gl-originating frames inside your own opengl
> canvas, then you need to develop some form of a gstreamer sink for
> your canvas.
>
> This is what has been done in both pigment (with pgmimagesink) and
> clutter (cluttergstsink).
>
> The clutter section in gst-gl examples directory shows a
> gst-gl-originating texture redirection inside a clutter application
> (using composite redirection).
>
>   
>> I'm wondering as this callback function is called by a
>> gstreamer thread and if the application is doing other OpenGL stuff or
>> contains more than one video display, multiple threads are doing
>> OpenGL drawing operations simultaneously.
>>     
>
> Your application should take care of the draws sync so that no
> conflict between opengl operations exists. Also, sharing opengl
> contexts between apps seems a difficult task.
>
>   
>> self.videosink = gst.element_factory_make('glimagesink', 'videosink')
>> self.videosink.set_property('client-draw-callback', self.draw_callback)
>>     
>
> I think the client-draw-callback will offer you a way to manipulate
> the video texture inside the glimagesink context, not the other way
> around (drawing video inside your own app).
>
> I assume you are only trying to display gst-originating (decoded
> video) video inside your Gl app. If so, you can prototype things using
> the fakesink element, by connecting it's handoff signals to a texture
> upload in your application. See [1] and [2] for more info
>
> Florent
>
> [1] http://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/section-data-spoof.html
> [2] http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-plugins/html/gstreamer-plugins-fakesink.html#GstFakeSink-handoff
>
> ------------------------------------------------------------------------------
> Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
> powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
> easily build your RIAs with Flex Builder, the Eclipse(TM)based development
> software that enables intelligent coding and step-through debugging.
> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>
>