using gl context of external app

Fri Sep 4 08:50:07 PDT 2015

Hi Matt,

Cool, thanks for the follow up on OS X. I'm pretty sure that JOGL doesn't
use XQuartz, so it is probably just a NSApp. For Linux and Windows, it
relies on X11
<https://github.com/sgothel/jogl/blob/master/src/nativewindow/native/x11/Xmisc.c#L489>
and (if understand the native core correctly) on the win32 API
<https://github.com/sgothel/jogl/blob/master/src/nativewindow/native/win32/GDImisc.c#L121>
.

Do you want me to open an issue on the gstreamer tracker requesting the API
enhancement, so at least the discussion is better focused?

Andres

On Fri, Sep 4, 2015 at 1:55 AM, Matthew Waters <ystreet00 at gmail.com> wrote:

> On 04/09/15 04:36, Andres Colubri wrote:
>
> It looks it will be a bit more involved that originally thought :-)
>
> I'm working on Mac right now, and the native window is not created neither
> with QT or GDK. Seems to be a straight NSWindow, handled by JOGL:
>
>
> https://github.com/sgothel/jogl/blob/master/src/nativewindow/native/macosx/OSXmisc.m#L342
>
> So I would imagine that I need to implement something analogous to your synchronous
> bus callback on X, using the corresponding NSWindow API to get the display,
> etc. This exceeds my current knowledge and available time, but will try to
> find a way to work through it.
>
>
> On OS X, you actually don't need to wrap a display object and can just use
> the default from gst_gl_display_new(), just the GL context (as long as JOGL
> uses NSApp I believe).  It's inferred from the global NSApp object provided
> by Cocoa.  If JOGL is actually using XQuartz and therefore X11, then you'll
> have to deal with the X11 display as you would on a Linux system.
>
> There's also a CoreAnimation layer CAOpenGLLayer that is available through
> GstGLCAOpenGLLayer that can be used if JOGL is using CALayer's instead.
> See the the sink at
> <http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gl/caopengllayersink.m>
> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gl/caopengllayersink.m
>
> Only out of curiosity, I wonder if it would be better to have a GstGL
> function that takes the handles to the display and the GL context, together
> with the window, platform, and gl api identifiers, and internally creates
> the GstContext wrapping the context from the app... but this is nothing
> more than moving the complexity from the user code into gstreamer, which is
> something you might not want to do/support, quite understandably.
>
>
> I have thought about adding that before and it still might be something we
> do to ease development.
>
> Cheers
> -Matt
>
>
> Thanks for all your help!
> Andres
>
>
>
>
> On Wed, Sep 2, 2015 at 10:47 AM, Andres Colubri <andres.colubri at gmail.com>
> wrote:
>
>> Hi Matt,
>>
>> Thanks a lot for your detailed answer! My application is in fact written
>> in Java, and uses JOGL to access the GL API, as well as to handle the
>> display across Mac, Windows, and Linux. I will see how can I get the native
>> handle to the display (getting the handle to the context is easy), and
>> apply the techniques in the code you linked to. I will post an update with
>> links to my source.
>>
>> Best,
>> Andres
>>
>> On Wed, Sep 2, 2015 at 10:37 AM, Matthew Waters < <ystreet00 at gmail.com>
>> ystreet00 at gmail.com> wrote:
>>
>>> On 02/09/15 18:20, Andres Colubri wrote:
>>> > Hi there,
>>> >
>>> > I have a question about the GL plugins. I'm implementing an OpenGL
>>> > application that uses gstreamer to play videos, and my goal is to have
>>> > gstreamer uploading the frames to GL textures, and the app rendering
>>> > these textres on its own GL surface. From what I read online, I should
>>> > use a glupload element to copy the frames into GL textures, and then
>>> > all I would need to do is to retrieve the glids using appsink. I'm
>>> > able to pull the ids with the following pipeline:
>>> >
>>> > uridecodebin uri=%s ! videoconvert ! " + caps + " ! videoscale !
>>> > glupload name=glup ! appsink name=sink
>>> >
>>> > However, in order to render the textures, gstreamer should use the
>>> > same GL context from my application.
>>> >
>>> > Fromt looking at the cluttershare example
>>> > (
>>> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tests/examples/gl/clutter/cluttershare.c
>>> )
>>> > and other pieces of code I found on the web, seems that I should pass
>>> > the handle of the GL context to gstreamer with the following code:
>>> >
>>> >   GstElement *glupload = gst_bin_get_by_name (GST_BIN (v->play),
>>> "glup");
>>> >   g_object_set (G_OBJECT (glupload), "external-opengl-context",
>>> > context, NULL);
>>> >   gst_object_unref (glupload);
>>> >
>>> > However I get the following error:
>>> >
>>> > (<unknown>:33272): GLib-GObject-WARNING **: g_object_set_valist:
>>> > object class 'GstGLUploadElement' has no property named
>>> > 'external-opengl-context'
>>> >
>>> > GstGLFilter seems to have an "other-context" property, but I don't
>>> > need to apply any filter on the frames, just to pull them out of the
>>> > pipeline. Any ideas on how to do this?
>>>
>>> Hi,
>>>
>>> Ok, it seems like you're attempting to use 1.4 API with a 1.5 gstreamer
>>> installation where the context sharing interface was mostly rewritten to
>>> use GstContext [1].
>>>
>>> Here's what you need to do:
>>> 1. Provide a display connection to GStreamer.  The easiest is to provide
>>> a GstGLDisplay wrapping your window system display connection as you'll
>>> need it later as well to create a wrapped GL context.  What exactly is
>>> required depends on the exact window system used. See [2] and [3] for
>>> some inspiration.
>>> 2. Provide the GL context to GStreamer as a GstGLContext.  GStreamer
>>> will then create it's own GL context that shares with the provided GL
>>> context.  See [4] and [5] for an example.
>>>
>>> When you provide these GstContext's mostly depends on your application
>>> but you essentially have two options as to when you call
>>> gst_element_set_context() with the required information (both display
>>> and GL context):
>>> 1. At application start up before the pipeline is running
>>> 2. As a response to a NEED_CONTEXT message you receive on the GstBus.
>>>
>>> See [6] for the current list of GstContext names currently accepted by
>>> all the GL elements.
>>>
>>> Cheers
>>> -Matt
>>>
>>> P.S your pipeline doesn't contain any video format conversion elements
>>> so whatever uridecodebin outputs will be pushed into appsink.  You can
>>> try adding glcolorconvert or videoconvert to convert to a format you
>>> want.
>>>
>>> [1] -
>>> https://developer.gnome.org/gstreamer/stable/gstreamer-GstContext.html
>>> [2] -
>>>
>>> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gtk/gtkgstglwidget.c#n421
>>> [3] -
>>>
>>> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/qt/qtitem.cc#n114
>>> [4] -
>>>
>>> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gtk/gtkgstglwidget.c#n471
>>> [5] -
>>>
>>> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/qt/qtitem.cc#n265
>>> [6] -
>>>
>>> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst-libs/gst/gl/gstglutils.c#n703
>>>
>>> Here's an example of a synchronous bus callback for an X11 display:
>>>
>>> static GstGLDisplay *gl_display = NULL;
>>> static Display *x11_display = NULL;
>>>
>>> static void
>>> _open_x11 ()
>>> {
>>>   if (!x11_display)
>>>     x11_display = XOpenDisplay (NULL);
>>> }
>>>
>>> static gboolean
>>> sync_bus_call (GstBus *bus, GstMessage *msg, gpointer    data)
>>> {
>>>   switch (GST_MESSAGE_TYPE (msg)) {
>>>     case GST_MESSAGE_NEED_CONTEXT:
>>>     {
>>>       const gchar *context_type;
>>>       GstContext *context = NULL;
>>>
>>>       gst_message_parse_context_type (msg, &context_type);
>>>       g_print("got need context %s\n", context_type);
>>>
>>>       if (g_strcmp0 (context_type, GST_GL_DISPLAY_CONTEXT_TYPE) == 0) {
>>>
>>>         _open_x11 ();
>>>
>>>         if (!gl_display)
>>>           gl_display = GST_GL_DISPLAY
>>> (gst_gl_display_x11_new_with_display (x11_display));
>>>
>>>         context = gst_context_new (GST_GL_DISPLAY_CONTEXT_TYPE, TRUE);
>>>         gst_context_set_gl_display (context, gl_display);
>>>
>>>         gst_element_set_context (GST_ELEMENT (msg->src), context);
>>>       }
>>>
>>>       /* Here you'd retrieve the GstGLContext like in [4] and [5] and do
>>> a
>>>        * similar thing as above with the display but matching types/names
>>>        * with the "gst.gl.app_context" case in [6] */
>>>
>>>       if (context)
>>>         gst_context_unref (context);
>>>       break;
>>>     }
>>>     default:
>>>       break;
>>>   }
>>>
>>>   return FALSE;
>>> }
>>>
>>> > Thanks!
>>> > Andres
>>> >
>>>
>>>
>>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20150904/f3d46394/attachment-0001.html>