gstreamer with cairo plugin

[Matthew Waters]

On 13/01/13 21:14, Sebastian Dröge wrote:
>>>> This would require the decoders to either be able to decode
>>>> directly
>>>> into GPU memory somehow or use GL... or it would be necessary for
>>>> the
>>>> cairo GL backend to be able to map the GPU memory to normal
>>>> system
>>>> memory for usage by other GStreamer elements.
>>> I believe he was thinking of extensions such as KHR_lock_surface[3],
>>> which
>>> "allows mapping color buffers of EGL surfaces into the client
>>> address space".
>>>
>>> Also, in the case of hardware decoders, I think we can often hope to
>>> get the
>>> video decoded into GPU memory. When such a case happens, we
>>> obviously want to
>>> keep buffers in the GPU until they are rendered.
>>> By the way, I am slightly confused: is there a "standardised" way
>>> (with a
>>> GstMeta I guess) in GStreamer 1.0 of saying things like "this buffer
>>> is in GPU
>>> memory" or "this function can give you this buffer mapped to an
>>> opengl(es)
>>> texture"?
>>> For instance, the metas in gstreamer-vaapi[2] only seem to provide
>>> info that is
>>> useful if you can "speak" vaapi.
>> No, there isn't a standardised way yet for notifying of GPU buffers
>> yet. [1] contains a basic attempt at some kind of standard
>> (GstGLMemory).  A more comprehensive approach should involve specific
>> GstMeta for both the GPU context and GPU buffers.  There also needs to
>> be a way of passing the GPU context between elements before buffers
>> are created (query/event?) if we are to implement context sharing.
> At least for Linux it would probably make sense to have a generic
> dma_buf meta that can be used to pass data between different devices
> directly. And a corresponding GstMemory/GstAllocator.

Agreed.

> The same should probably be done for EGLImages.

EGLImages are viable but alas are EGL only.  And while EGL should rule 
the world there is still WGL, CGL/AGL and GLX (arguably soon to be 
deprecated) to deal with.

> Putting generic GL
> "resources" into buffers is IMHO calling for trouble as it's not
> guaranteed at all that they can be used in different threads.

That is essentially the reason why gst-plugins-gl has a seperate opengl 
thread.  An alternative is context sharing (either per thread or per 
element).  Attempting to do anything else with opengl gets complicated 
way too fast.

-- 
-Matt