<html> <head> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> </head> <body text="#000000" bgcolor="#FFFFFF"> <div class="moz-text-flowed" style="font-family: -moz-fixed; font-size: 12px;" lang="x-western">On 12/01/13 04:59, Guillaume Emont wrote: <blockquote type="cite" style="color: #000000;">On Fri, Jan 11, 2013 at 03:33:22PM +0100, Sebastian Dröge wrote: <blockquote type="cite" style="color: #000000;">On Do, 2013-01-10 at 22:18 +0000, Henry (Yu) Song - SISA wrote: <blockquote type="cite" style="color: #000000;">Well First, we would like to update cairo plugin such that it is up-to-date, and maintained. Second we like to to use cairo_gl_surface instead of image surface. As far as I know our cairo-gles backend is the fastest one among all cairo backend. It is maintained by me and is being actively optimized/bug fixed. </blockquote> Which parts of the cairo plugin would this be? It contains (contained) multiple elements. </blockquote> I think that the most important goal is to have video playback well integrated in an interface that uses cairo. So I believe that a sink is the part most needed. Not sure there ever was a cairo sink previously, apart from Benjamin Otte's experiment[1], or was there? <blockquote type="cite" style="color: #000000;">Switching it unconditionally to GL doesn't sound like a good idea though, it should only use GL if GL is used elsewhere in the pipeline. Otherwise the overhead of uploading/downloading to/from the GPU is probably too high. But this can be done, is mostly unrelated to cairo though. </blockquote> Yes, ideally, we should be able to: - know whether cairo is using gl or not (well, rather easy) - if we use gl, be able to negotiate with upstream whether to use buffers in GPU memory or in CPU memory I am currently looking at various other sinks that seem to try to do that or similar things, such as gst-vaapi[2]. Are there other examples out there where I should get inspiration? </blockquote> You may want to look at [1] as well. <blockquote type="cite" style="color: #000000;"> <blockquote type="cite" style="color: #000000;"> <blockquote type="cite" style="color: #000000;">Second, I would like to decode image into GPU memory directly with cairo gl backend as sink, such that we can either play it back much quicker instead of upload to texture and we can add overlay, which may also be in GPU memory, quicker. </blockquote> This would require the decoders to either be able to decode directly into GPU memory somehow or use GL... or it would be necessary for the cairo GL backend to be able to map the GPU memory to normal system memory for usage by other GStreamer elements. </blockquote> I believe he was thinking of extensions such as KHR_lock_surface[3], which "allows mapping color buffers of EGL surfaces into the client address space". Also, in the case of hardware decoders, I think we can often hope to get the video decoded into GPU memory. When such a case happens, we obviously want to keep buffers in the GPU until they are rendered. By the way, I am slightly confused: is there a "standardised" way (with a GstMeta I guess) in GStreamer 1.0 of saying things like "this buffer is in GPU memory" or "this function can give you this buffer mapped to an opengl(es) texture"? For instance, the metas in gstreamer-vaapi[2] only seem to provide info that is useful if you can "speak" vaapi. </blockquote> No, there isn't a standardised way yet for notifying of GPU buffers yet. [1] contains a basic attempt at some kind of standard (GstGLMemory). A more comprehensive approach should involve specific GstMeta for both the GPU context and GPU buffers. There also needs to be a way of passing the GPU context between elements before buffers are created (query/event?) if we are to implement context sharing. Mapping a buffer may give you opengl(es) texture data. The question is where you want the data to be, GPU or Main memory. <blockquote type="cite" style="color: #000000;">Guillaume [1] <a class="moz-txt-link-freetext" href="http://cgit.freedesktop.org/%7Ecompany/gst-plugins-cairo/">http://cgit.freedesktop.org/~company/gst-plugins-cairo/</a> [2] <a class="moz-txt-link-freetext" href="http://cgit.freedesktop.org/%7Esree/gstreamer-vaapi-1.0/">http://cgit.freedesktop.org/~sree/gstreamer-vaapi-1.0/</a> [3] <a class="moz-txt-link-freetext" href="http://www.khronos.org/registry/egl/extensions/KHR/EGL_KHR_lock_surface.txt">http://www.khronos.org/registry/egl/extensions/KHR/EGL_KHR_lock_surface.txt</a> <blockquote type="cite" style="color: #000000;">_______________________________________________ gstreamer-devel mailing list <a class="moz-txt-link-abbreviated" href="mailto:gstreamer-devel@lists.freedesktop.org">gstreamer-devel@lists.freedesktop.org</a> <a class="moz-txt-link-freetext" href="http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel">http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel</a> </blockquote> </blockquote> [1] <a class="moz-txt-link-freetext" href="https://github.com/ystreet/gst-plugins-gl">https://github.com/ystreet/gst-plugins-gl</a> <div class="moz-txt-sig">-- -Matt </div> </div> </body> </html>