Hi, <div class="gmail_quote">2009/3/24 Andres Colubri <<a href="mailto:andres.colubri@gmail.com">andres.colubri@gmail.com</a>> <blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> Hi, I have been following this discussion with interest and I just wanted to add a couple of comments/questions. I'm currently combining opengl and gstreamer within my application. Right now, the gst-originated video buffers are pushed out of the pipeline with a handoff signal, and then uploaded to an opengl texture for further gpu-accelerated processing and final display on the application window.</blockquote><div>ok </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> But it would be quite great if I can just get the gl texture id from the gst-plugins, for instance glupload, and then directly use the texture inside my application.</blockquote><div> Would be cool but it's not possible for the reasons you explain next. (a texture id can be used only inside the opengl context where it comes from) But there is maybe an other way: --on win32: wglCopyContext(glcontextSrc, glcontextDst, GL_TEXTURE_BIT); --on linux (assume it's X): glXCopyContext(disp, glcontextSrc, glcontextDst, GL_TEXTURE_BIT); --on OpenGL ES 2.0: <a href="http://www.khronos.org/opengles/sdk/1.1/docs/man/eglCreateContext.xml">http://www.khronos.org/opengles/sdk/1.1/docs/man/eglCreateContext.xml</a> (Specifies the EGL rendering context with which to share texture objects) I have not experimented those things a lot. So I cannot tell more about this possible way. (there is some restrictions but in our case we are ok) So we can try it, for example I enable a function from a glfilter that would allow to retrieve the glcontext used inside the pipeline. You could then just call glCopyContext one time, and then use the texture id(s) just like you want. </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> Since my app is cross-platform (linux, win, osx), the composite redirection approach doesn't seem viable. The XOverlay interface won't work either, because I need to do some gpgpu post-processing before the actual image rendering.</blockquote><div> </div><div>Yup, the XOverlay interface is only there to specify a window id. There is not any relation with OpenGL (I mean about the goal). </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> >From what I read, one alternative (perhaps the only one under this type of scenarios) would be to share the opengl context between the app and the gst gl-plugin. Basically, the app would create the opengl context and then pass it to the plugin when initializing the gst pipeline.</blockquote><div> This is a solution (use the same opengl context bettween you app and the plugin). I also first added a function but it's currently not implemented because of time (and I think there is a better/prefered way). <a href="http://cgit.freedesktop.org/gstreamer/gst-plugins-gl/tree/gst-libs/gst/gl/gstglwindow.h">http://cgit.freedesktop.org/gstreamer/gst-plugins-gl/tree/gst-libs/gst/gl/gstglwindow.h</a> <a href="http://cgit.freedesktop.org/gstreamer/gst-plugins-gl/tree/gst-libs/gst/gl">http://cgit.freedesktop.org/gstreamer/gst-plugins-gl/tree/gst-libs/gst/gl</a> The main bad thing would be to use a lot of glMakeCurrent call (for each frame), inside your app thread and inside the gst-gl thread. For now, there is only one call to glMakeCurrent at gst-gl init. Switching a gl context bettween different threads it's very expensive. More over with several opengl context (several branch in the pipeline) this solution would be like a hell. I have experience on this so I can tell more and that's a bad way. </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> Is this possible in the current implementation of gst-plugins? If not, wouldn't be an useful (additional) method for setting up gl-accelerated pipelines?</blockquote><div> There is 2 other solutions. First, have you take a look at glfilterapp ? Look at the code of gst-plugins-gl/tests/examples/generic/recordgraphic/main.cpp <div>There is a drawcallback, which is invoked each time a new video frame is upload, and at the point of glfilterapp place in the graph. This is the easiest way, if your scene is not extremelly complex. In this way, you have not to create/handle the opengl context, but you can give a windows ID in which you want see the gl scene. </div>The last solution is to write a glfilter as glbumper, gleffects, glfiltercube, glfiltersobel (etc..) are, and use you animation API inside the implementation of its. Also you could control your filter through gobject properties as it's done for gleffects (see property "effect"). And it would be more flexible. Finally, you could also split your rendering process in several glfilters. For example, one for the scene (rotation, quad), and one other for the effect on the texture. In this way too, you have not to create/handle the opengl context, but you can give a windows ID in which you want see the gl scene. I think it's the best way, the more viable, useable, flexible. So we have 4 solutions. The 3 (glfilterapp), and 4 (create a glfilter) are already fully implemented. The 1 (copyContext), we have to test it. The 2, (use the same context) must be used only if 1, 3 and 4 are not possible (because really a bad way). And it's not yet implemented. About the solution 4, you will tell me that you can implement a glfilter because you are using a framework that comes with its own opengl context management. So the problem is still the same. (I mean, all the opengl framework focused on draw and esthetic: clutter etc...) Then I will say yes, but then I will tell you that the solution is to implement glwindow with your friendly framework. I am mean if it's clutter, then you have to create a gstglwindow_clutter.c. I can help on this. </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> Thanks, Andres</blockquote><div> I hope this can help. Sincerely Julien </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> <div><div></div><div class="h5"> Florent wrote: > Hi, > > >> Thanks Jan, I installed gst-plugins-gl from git and did some first >> experiments. This seems to be what I'm looking for. Is it correct that >> I would use 'glimagesink' and do the drawing using the >> client-draw-callback? I'm not experienced with OpenGL, is it safe to >> just redraw the part of the scene that shows the video in the >> callback? >> > > If i'm not mistaken glimagesink isn't what you are looking for, except > if your opengl application supports composite redirection (when > glimagesink's textures are redirected inside your own opengl app): the > glimagesink will be a dedicated, separate window and gl context. If > you need to display gst-gl-originating frames inside your own opengl > canvas, then you need to develop some form of a gstreamer sink for > your canvas. > > This is what has been done in both pigment (with pgmimagesink) and > clutter (cluttergstsink). > > The clutter section in gst-gl examples directory shows a > gst-gl-originating texture redirection inside a clutter application > (using composite redirection). > > >> I'm wondering as this callback function is called by a >> gstreamer thread and if the application is doing other OpenGL stuff or >> contains more than one video display, multiple threads are doing >> OpenGL drawing operations simultaneously. >> > > Your application should take care of the draws sync so that no > conflict between opengl operations exists. Also, sharing opengl > contexts between apps seems a difficult task. > > >> self.videosink = gst.element_factory_make('glimagesink', 'videosink') >> self.videosink.set_property('client-draw-callback', self.draw_callback) >> > > I think the client-draw-callback will offer you a way to manipulate > the video texture inside the glimagesink context, not the other way > around (drawing video inside your own app). > > I assume you are only trying to display gst-originating (decoded > video) video inside your Gl app. If so, you can prototype things using > the fakesink element, by connecting it's handoff signals to a texture > upload in your application. See [1] and [2] for more info > > Florent > > [1] <a href="http://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/section-data-spoof.html" target="_blank">http://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/section-data-spoof.html</a> > [2] <a href="http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-plugins/html/gstreamer-plugins-fakesink.html#GstFakeSink-handoff" target="_blank">http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-plugins/html/gstreamer-plugins-fakesink.html#GstFakeSink-handoff</a> > > ------------------------------------------------------------------------------ > Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are > powering Web 2.0 with engaging, cross-platform capabilities. Quickly and > easily build your RIAs with Flex Builder, the Eclipse(TM)based development > software that enables intelligent coding and step-through debugging. > Download the free 60 day trial. <a href="http://p.sf.net/sfu/www-adobe-com" target="_blank">http://p.sf.net/sfu/www-adobe-com</a> > _______________________________________________ > gstreamer-devel mailing list > <a href="mailto:gstreamer-devel@lists.sourceforge.net">gstreamer-devel@lists.sourceforge.net</a> > <a href="https://lists.sourceforge.net/lists/listinfo/gstreamer-devel" target="_blank">https://lists.sourceforge.net/lists/listinfo/gstreamer-devel</a> > > ------------------------------------------------------------------------------ Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. <a href="http://p.sf.net/sfu/www-adobe-com" target="_blank">http://p.sf.net/sfu/www-adobe-com</a> _______________________________________________ gstreamer-devel mailing list <a href="mailto:gstreamer-devel@lists.sourceforge.net">gstreamer-devel@lists.sourceforge.net</a> <a href="https://lists.sourceforge.net/lists/listinfo/gstreamer-devel" target="_blank">https://lists.sourceforge.net/lists/listinfo/gstreamer-devel</a> </div></div></blockquote></div>