using gl context of external app

Thu Sep 3 11:36:16 PDT 2015

It looks it will be a bit more involved that originally thought :-)

I'm working on Mac right now, and the native window is not created neither
with QT or GDK. Seems to be a straight NSWindow, handled by JOGL:

https://github.com/sgothel/jogl/blob/master/src/nativewindow/native/macosx/OSXmisc.m#L342

So I would imagine that I need to implement something analogous to
your synchronous
bus callback on X, using the corresponding NSWindow API to get the display,
etc. This exceeds my current knowledge and available time, but will try to
find a way to work through it.

Only out of curiosity, I wonder if it would be better to have a GstGL
function that takes the handles to the display and the GL context, together
with the window, platform, and gl api identifiers, and internally creates
the GstContext wrapping the context from the app... but this is nothing
more than moving the complexity from the user code into gstreamer, which is
something you might not want to do/support, quite understandably.

Thanks for all your help!
Andres

On Wed, Sep 2, 2015 at 10:47 AM, Andres Colubri <andres.colubri at gmail.com>
wrote:

> Hi Matt,
>
> Thanks a lot for your detailed answer! My application is in fact written
> in Java, and uses JOGL to access the GL API, as well as to handle the
> display across Mac, Windows, and Linux. I will see how can I get the native
> handle to the display (getting the handle to the context is easy), and
> apply the techniques in the code you linked to. I will post an update with
> links to my source.
>
> Best,
> Andres
>
> On Wed, Sep 2, 2015 at 10:37 AM, Matthew Waters <ystreet00 at gmail.com>
> wrote:
>
>> On 02/09/15 18:20, Andres Colubri wrote:
>> > Hi there,
>> >
>> > I have a question about the GL plugins. I'm implementing an OpenGL
>> > application that uses gstreamer to play videos, and my goal is to have
>> > gstreamer uploading the frames to GL textures, and the app rendering
>> > these textres on its own GL surface. From what I read online, I should
>> > use a glupload element to copy the frames into GL textures, and then
>> > all I would need to do is to retrieve the glids using appsink. I'm
>> > able to pull the ids with the following pipeline:
>> >
>> > uridecodebin uri=%s ! videoconvert ! " + caps + " ! videoscale !
>> > glupload name=glup ! appsink name=sink
>> >
>> > However, in order to render the textures, gstreamer should use the
>> > same GL context from my application.
>> >
>> > Fromt looking at the cluttershare example
>> > (
>> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tests/examples/gl/clutter/cluttershare.c
>> )
>> > and other pieces of code I found on the web, seems that I should pass
>> > the handle of the GL context to gstreamer with the following code:
>> >
>> >   GstElement *glupload = gst_bin_get_by_name (GST_BIN (v->play),
>> "glup");
>> >   g_object_set (G_OBJECT (glupload), "external-opengl-context",
>> > context, NULL);
>> >   gst_object_unref (glupload);
>> >
>> > However I get the following error:
>> >
>> > (<unknown>:33272): GLib-GObject-WARNING **: g_object_set_valist:
>> > object class 'GstGLUploadElement' has no property named
>> > 'external-opengl-context'
>> >
>> > GstGLFilter seems to have an "other-context" property, but I don't
>> > need to apply any filter on the frames, just to pull them out of the
>> > pipeline. Any ideas on how to do this?
>>
>> Hi,
>>
>> Ok, it seems like you're attempting to use 1.4 API with a 1.5 gstreamer
>> installation where the context sharing interface was mostly rewritten to
>> use GstContext [1].
>>
>> Here's what you need to do:
>> 1. Provide a display connection to GStreamer.  The easiest is to provide
>> a GstGLDisplay wrapping your window system display connection as you'll
>> need it later as well to create a wrapped GL context.  What exactly is
>> required depends on the exact window system used. See [2] and [3] for
>> some inspiration.
>> 2. Provide the GL context to GStreamer as a GstGLContext.  GStreamer
>> will then create it's own GL context that shares with the provided GL
>> context.  See [4] and [5] for an example.
>>
>> When you provide these GstContext's mostly depends on your application
>> but you essentially have two options as to when you call
>> gst_element_set_context() with the required information (both display
>> and GL context):
>> 1. At application start up before the pipeline is running
>> 2. As a response to a NEED_CONTEXT message you receive on the GstBus.
>>
>> See [6] for the current list of GstContext names currently accepted by
>> all the GL elements.
>>
>> Cheers
>> -Matt
>>
>> P.S your pipeline doesn't contain any video format conversion elements
>> so whatever uridecodebin outputs will be pushed into appsink.  You can
>> try adding glcolorconvert or videoconvert to convert to a format you want.
>>
>> [1] -
>> https://developer.gnome.org/gstreamer/stable/gstreamer-GstContext.html
>> [2] -
>>
>> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gtk/gtkgstglwidget.c#n421
>> [3] -
>>
>> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/qt/qtitem.cc#n114
>> [4] -
>>
>> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gtk/gtkgstglwidget.c#n471
>> [5] -
>>
>> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/qt/qtitem.cc#n265
>> [6] -
>>
>> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst-libs/gst/gl/gstglutils.c#n703
>>
>> Here's an example of a synchronous bus callback for an X11 display:
>>
>> static GstGLDisplay *gl_display = NULL;
>> static Display *x11_display = NULL;
>>
>> static void
>> _open_x11 ()
>> {
>>   if (!x11_display)
>>     x11_display = XOpenDisplay (NULL);
>> }
>>
>> static gboolean
>> sync_bus_call (GstBus *bus, GstMessage *msg, gpointer    data)
>> {
>>   switch (GST_MESSAGE_TYPE (msg)) {
>>     case GST_MESSAGE_NEED_CONTEXT:
>>     {
>>       const gchar *context_type;
>>       GstContext *context = NULL;
>>
>>       gst_message_parse_context_type (msg, &context_type);
>>       g_print("got need context %s\n", context_type);
>>
>>       if (g_strcmp0 (context_type, GST_GL_DISPLAY_CONTEXT_TYPE) == 0) {
>>
>>         _open_x11 ();
>>
>>         if (!gl_display)
>>           gl_display = GST_GL_DISPLAY
>> (gst_gl_display_x11_new_with_display (x11_display));
>>
>>         context = gst_context_new (GST_GL_DISPLAY_CONTEXT_TYPE, TRUE);
>>         gst_context_set_gl_display (context, gl_display);
>>
>>         gst_element_set_context (GST_ELEMENT (msg->src), context);
>>       }
>>
>>       /* Here you'd retrieve the GstGLContext like in [4] and [5] and do a
>>        * similar thing as above with the display but matching types/names
>>        * with the "gst.gl.app_context" case in [6] */
>>
>>       if (context)
>>         gst_context_unref (context);
>>       break;
>>     }
>>     default:
>>       break;
>>   }
>>
>>   return FALSE;
>> }
>>
>> > Thanks!
>> > Andres
>> >
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20150903/73112287/attachment-0001.html>