gstreamer with cairo plugin

[Matthew Waters]

On 12/01/13 04:59, Guillaume Emont wrote:
> On Fri, Jan 11, 2013 at 03:33:22PM +0100, Sebastian Dröge wrote:
>> On Do, 2013-01-10 at 22:18 +0000, Henry (Yu) Song - SISA wrote:
>>> Well
>>>
>>> First, we would like to update cairo plugin such that it is up-to-date,
>>> and maintained.  Second we like to to use cairo_gl_surface instead of
>>> image surface.  As far as I know our cairo-gles backend is the fastest
>>> one among all cairo backend.  It is maintained by me and is being
>>> actively optimized/bug fixed.
>> Which parts of the cairo plugin would this be? It contains (contained)
>> multiple elements.
> I think that the most important goal is to have video playback well 
> integrated
> in an interface that uses cairo. So I believe that a sink is the part 
> most
> needed. Not sure there ever was a cairo sink previously, apart from 
> Benjamin
> Otte's experiment[1], or was there?
>
>> Switching it unconditionally to GL doesn't sound like a good idea
>> though, it should only use GL if GL is used elsewhere in the pipeline.
>> Otherwise the overhead of uploading/downloading to/from the GPU is
>> probably too high.
>> But this can be done, is mostly unrelated to cairo though.
> Yes, ideally, we should be able to:
>   - know whether cairo is using gl or not (well, rather easy)
>   - if we use gl, be able to negotiate with upstream whether to use 
> buffers in
>     GPU memory or in CPU memory
>
> I am currently looking at various other sinks that seem to try to do 
> that or
> similar things, such as gst-vaapi[2]. Are there other examples out 
> there where
> I should get inspiration?
You may want to look at [1] as well.
>>> Second, I would like to decode image into GPU memory directly with 
>>> cairo
>>> gl backend as sink, such that we can either play it back much quicker
>>> instead of upload to texture and we can add overlay, which may also be
>>> in GPU memory, quicker.
>> This would require the decoders to either be able to decode directly
>> into GPU memory somehow or use GL... or it would be necessary for the
>> cairo GL backend to be able to map the GPU memory to normal system
>> memory for usage by other GStreamer elements.
> I believe he was thinking of extensions such as KHR_lock_surface[3], 
> which
> "allows mapping color buffers of EGL surfaces into the client address 
> space".
>
> Also, in the case of hardware decoders, I think we can often hope to 
> get the
> video decoded into GPU memory. When such a case happens, we obviously 
> want to
> keep buffers in the GPU until they are rendered.
> By the way, I am slightly confused: is there a "standardised" way (with a
> GstMeta I guess) in GStreamer 1.0 of saying things like "this buffer 
> is in GPU
> memory" or "this function can give you this buffer mapped to an 
> opengl(es)
> texture"?
> For instance, the metas in gstreamer-vaapi[2] only seem to provide 
> info that is
> useful if you can "speak" vaapi.
No, there isn't a standardised way yet for notifying of GPU buffers yet. 
[1] contains a basic attempt at some kind of standard (GstGLMemory).  A 
more comprehensive approach should involve specific GstMeta for both the 
GPU context and GPU buffers.  There also needs to be a way of passing 
the GPU context between elements before buffers are created 
(query/event?) if we are to implement context sharing.

Mapping a buffer may give you opengl(es) texture data.  The question is 
where you want the data to be, GPU or Main memory.
> Guillaume
>
> [1] http://cgit.freedesktop.org/~company/gst-plugins-cairo/
> [2] http://cgit.freedesktop.org/~sree/gstreamer-vaapi-1.0/
> [3] 
> http://www.khronos.org/registry/egl/extensions/KHR/EGL_KHR_lock_surface.txt 
>
>
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
[1] https://github.com/ystreet/gst-plugins-gl

-- 
-Matt

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20130113/be4d79b4/attachment.html>