using gl context of external app

[Matthew Waters]

On 02/09/15 18:20, Andres Colubri wrote:
> Hi there,
>
> I have a question about the GL plugins. I'm implementing an OpenGL
> application that uses gstreamer to play videos, and my goal is to have
> gstreamer uploading the frames to GL textures, and the app rendering
> these textres on its own GL surface. From what I read online, I should
> use a glupload element to copy the frames into GL textures, and then
> all I would need to do is to retrieve the glids using appsink. I'm
> able to pull the ids with the following pipeline:
>
> uridecodebin uri=%s ! videoconvert ! " + caps + " ! videoscale !
> glupload name=glup ! appsink name=sink
>
> However, in order to render the textures, gstreamer should use the
> same GL context from my application. 
>
> Fromt looking at the cluttershare example
> (http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tests/examples/gl/clutter/cluttershare.c)
> and other pieces of code I found on the web, seems that I should pass
> the handle of the GL context to gstreamer with the following code:
>
>   GstElement *glupload = gst_bin_get_by_name (GST_BIN (v->play), "glup");
>   g_object_set (G_OBJECT (glupload), "external-opengl-context",
> context, NULL);
>   gst_object_unref (glupload);
>
> However I get the following error:
>
> (<unknown>:33272): GLib-GObject-WARNING **: g_object_set_valist:
> object class 'GstGLUploadElement' has no property named
> 'external-opengl-context'
>
> GstGLFilter seems to have an "other-context" property, but I don't
> need to apply any filter on the frames, just to pull them out of the
> pipeline. Any ideas on how to do this?

Hi,

Ok, it seems like you're attempting to use 1.4 API with a 1.5 gstreamer
installation where the context sharing interface was mostly rewritten to
use GstContext [1].

Here's what you need to do:
1. Provide a display connection to GStreamer.  The easiest is to provide
a GstGLDisplay wrapping your window system display connection as you'll
need it later as well to create a wrapped GL context.  What exactly is
required depends on the exact window system used. See [2] and [3] for
some inspiration.
2. Provide the GL context to GStreamer as a GstGLContext.  GStreamer
will then create it's own GL context that shares with the provided GL
context.  See [4] and [5] for an example.

When you provide these GstContext's mostly depends on your application
but you essentially have two options as to when you call
gst_element_set_context() with the required information (both display
and GL context):
1. At application start up before the pipeline is running
2. As a response to a NEED_CONTEXT message you receive on the GstBus.

See [6] for the current list of GstContext names currently accepted by
all the GL elements.

Cheers
-Matt

P.S your pipeline doesn't contain any video format conversion elements
so whatever uridecodebin outputs will be pushed into appsink.  You can
try adding glcolorconvert or videoconvert to convert to a format you want.

[1] - https://developer.gnome.org/gstreamer/stable/gstreamer-GstContext.html
[2] -
http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gtk/gtkgstglwidget.c#n421
[3] -
http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/qt/qtitem.cc#n114
[4] -
http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gtk/gtkgstglwidget.c#n471
[5] -
http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/qt/qtitem.cc#n265
[6] -
http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst-libs/gst/gl/gstglutils.c#n703

Here's an example of a synchronous bus callback for an X11 display:

static GstGLDisplay *gl_display = NULL;
static Display *x11_display = NULL;

static void
_open_x11 ()
{
  if (!x11_display)
    x11_display = XOpenDisplay (NULL);
}

static gboolean
sync_bus_call (GstBus *bus, GstMessage *msg, gpointer    data)
{
  switch (GST_MESSAGE_TYPE (msg)) {
    case GST_MESSAGE_NEED_CONTEXT:
    {
      const gchar *context_type;
      GstContext *context = NULL;
     
      gst_message_parse_context_type (msg, &context_type);
      g_print("got need context %s\n", context_type);

      if (g_strcmp0 (context_type, GST_GL_DISPLAY_CONTEXT_TYPE) == 0) {
     
        _open_x11 ();

        if (!gl_display)
          gl_display = GST_GL_DISPLAY
(gst_gl_display_x11_new_with_display (x11_display));

        context = gst_context_new (GST_GL_DISPLAY_CONTEXT_TYPE, TRUE);
        gst_context_set_gl_display (context, gl_display);

        gst_element_set_context (GST_ELEMENT (msg->src), context);
      }

      /* Here you'd retrieve the GstGLContext like in [4] and [5] and do a
       * similar thing as above with the display but matching types/names
       * with the "gst.gl.app_context" case in [6] */

      if (context)
        gst_context_unref (context);
      break;
    }
    default:
      break;
  }

  return FALSE;
}

> Thanks!
> Andres
>


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20150903/e8277799/attachment.sig>