gst-vaapi and opengl

[Nicolas Dufresne]

Le lundi 09 juillet 2012 à 06:03 -0700, leo yvin a écrit : 
> Hi,
> 
> I'm trying to use gst-vaapi to render a video in an OpenGl texture but i
> don't understand how it works. 
> 
> I can play a video with this pipeline : filesrc -> qtdemux -> vaapidecode ->
> vaapisink
> 
> and I can upload/display a texture in an openGl context with a 
> GstVaapiDisplay/GstVaapiSurface/...
> 
> but how can i link these two parts? 
> I didn't find an example, can someone give me one?

Hi Léo,

the vaapidecode element will produce buffers witch implement
GstSurfaceBuffer. This interface (part of gst-plugins-bad) let you
create a converter that will upload to the texture you want. To do so,
you may use the GstAppSink or implement your own video sink. When you
receive the first buffer, you will create a converter of type
"opengl" (the only supported type at the moment) and the ID of your
destination texture. Note that whenever the capabilities changes, you
need to renew the converter. Finally, call the upload method to upload
the new buffer to your texture.

Here is an exemple extracted from clutter-gst project: 
        static void
        clutter_gst_hw_upload (ClutterGstVideoSink *sink,
                               GstBuffer           *buffer)
        {
          ClutterGstVideoSinkPrivate *priv = sink->priv;
          GstSurfaceBuffer *surface;
        
          g_return_if_fail (GST_IS_SURFACE_BUFFER (buffer));
          surface = GST_SURFACE_BUFFER (buffer);
        
          if (G_UNLIKELY (priv->converter == NULL)) {
            CoglHandle tex;
            GLuint gl_texture;
            GLenum gl_target;
            GValue value = {0};
        
            tex = clutter_texture_get_cogl_texture (priv->texture);
            cogl_texture_get_gl_texture (tex, &gl_texture, &gl_target);
            g_return_if_fail (gl_target == GL_TEXTURE_2D);
        
            g_value_init (&value, G_TYPE_UINT);
            g_value_set_uint (&value, gl_texture);
        
            priv->converter = gst_surface_buffer_create_converter (surface, "opengl", &value);
            g_return_if_fail (priv->converter);
          }
        
          gst_surface_converter_upload (priv->converter, surface);
        
          /* The texture is dirty, schedule a redraw */
          clutter_actor_queue_redraw (CLUTTER_ACTOR (priv->texture));
        }

cheers,
Nicolas