using gl context of external app
Matthew Waters
ystreet00 at gmail.com
Thu Sep 3 22:55:52 PDT 2015
On 04/09/15 04:36, Andres Colubri wrote:
> It looks it will be a bit more involved that originally thought :-)
>
> I'm working on Mac right now, and the native window is not created
> neither with QT or GDK. Seems to be a straight NSWindow, handled by JOGL:
>
> https://github.com/sgothel/jogl/blob/master/src/nativewindow/native/macosx/OSXmisc.m#L342
>
> So I would imagine that I need to implement something analogous to
> your synchronous bus callback on X, using the corresponding NSWindow
> API to get the display, etc. This exceeds my current knowledge and
> available time, but will try to find a way to work through it.
On OS X, you actually don't need to wrap a display object and can just
use the default from gst_gl_display_new(), just the GL context (as long
as JOGL uses NSApp I believe). It's inferred from the global NSApp
object provided by Cocoa. If JOGL is actually using XQuartz and
therefore X11, then you'll have to deal with the X11 display as you
would on a Linux system.
There's also a CoreAnimation layer CAOpenGLLayer that is available
through GstGLCAOpenGLLayer that can be used if JOGL is using CALayer's
instead. See the the sink at
http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gl/caopengllayersink.m
> Only out of curiosity, I wonder if it would be better to have a GstGL
> function that takes the handles to the display and the GL context,
> together with the window, platform, and gl api identifiers, and
> internally creates the GstContext wrapping the context from the app...
> but this is nothing more than moving the complexity from the user code
> into gstreamer, which is something you might not want to do/support,
> quite understandably.
I have thought about adding that before and it still might be something
we do to ease development.
Cheers
-Matt
> Thanks for all your help!
> Andres
>
>
>
>
> On Wed, Sep 2, 2015 at 10:47 AM, Andres Colubri
> <andres.colubri at gmail.com <mailto:andres.colubri at gmail.com>> wrote:
>
> Hi Matt,
>
> Thanks a lot for your detailed answer! My application is in fact
> written in Java, and uses JOGL to access the GL API, as well as to
> handle the display across Mac, Windows, and Linux. I will see how
> can I get the native handle to the display (getting the handle to
> the context is easy), and apply the techniques in the code you
> linked to. I will post an update with links to my source.
>
> Best,
> Andres
>
> On Wed, Sep 2, 2015 at 10:37 AM, Matthew Waters
> <ystreet00 at gmail.com <mailto:ystreet00 at gmail.com>> wrote:
>
> On 02/09/15 18:20, Andres Colubri wrote:
> > Hi there,
> >
> > I have a question about the GL plugins. I'm implementing an
> OpenGL
> > application that uses gstreamer to play videos, and my goal
> is to have
> > gstreamer uploading the frames to GL textures, and the app
> rendering
> > these textres on its own GL surface. From what I read
> online, I should
> > use a glupload element to copy the frames into GL textures,
> and then
> > all I would need to do is to retrieve the glids using
> appsink. I'm
> > able to pull the ids with the following pipeline:
> >
> > uridecodebin uri=%s ! videoconvert ! " + caps + " ! videoscale !
> > glupload name=glup ! appsink name=sink
> >
> > However, in order to render the textures, gstreamer should
> use the
> > same GL context from my application.
> >
> > Fromt looking at the cluttershare example
> >
> (http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tests/examples/gl/clutter/cluttershare.c)
> > and other pieces of code I found on the web, seems that I
> should pass
> > the handle of the GL context to gstreamer with the following
> code:
> >
> > GstElement *glupload = gst_bin_get_by_name (GST_BIN
> (v->play), "glup");
> > g_object_set (G_OBJECT (glupload), "external-opengl-context",
> > context, NULL);
> > gst_object_unref (glupload);
> >
> > However I get the following error:
> >
> > (<unknown>:33272): GLib-GObject-WARNING **: g_object_set_valist:
> > object class 'GstGLUploadElement' has no property named
> > 'external-opengl-context'
> >
> > GstGLFilter seems to have an "other-context" property, but I
> don't
> > need to apply any filter on the frames, just to pull them
> out of the
> > pipeline. Any ideas on how to do this?
>
> Hi,
>
> Ok, it seems like you're attempting to use 1.4 API with a 1.5
> gstreamer
> installation where the context sharing interface was mostly
> rewritten to
> use GstContext [1].
>
> Here's what you need to do:
> 1. Provide a display connection to GStreamer. The easiest is
> to provide
> a GstGLDisplay wrapping your window system display connection
> as you'll
> need it later as well to create a wrapped GL context. What
> exactly is
> required depends on the exact window system used. See [2] and
> [3] for
> some inspiration.
> 2. Provide the GL context to GStreamer as a GstGLContext.
> GStreamer
> will then create it's own GL context that shares with the
> provided GL
> context. See [4] and [5] for an example.
>
> When you provide these GstContext's mostly depends on your
> application
> but you essentially have two options as to when you call
> gst_element_set_context() with the required information (both
> display
> and GL context):
> 1. At application start up before the pipeline is running
> 2. As a response to a NEED_CONTEXT message you receive on the
> GstBus.
>
> See [6] for the current list of GstContext names currently
> accepted by
> all the GL elements.
>
> Cheers
> -Matt
>
> P.S your pipeline doesn't contain any video format conversion
> elements
> so whatever uridecodebin outputs will be pushed into appsink.
> You can
> try adding glcolorconvert or videoconvert to convert to a
> format you want.
>
> [1] -
> https://developer.gnome.org/gstreamer/stable/gstreamer-GstContext.html
> [2] -
> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gtk/gtkgstglwidget.c#n421
> [3] -
> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/qt/qtitem.cc#n114
> [4] -
> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gtk/gtkgstglwidget.c#n471
> [5] -
> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/qt/qtitem.cc#n265
> [6] -
> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst-libs/gst/gl/gstglutils.c#n703
>
> Here's an example of a synchronous bus callback for an X11
> display:
>
> static GstGLDisplay *gl_display = NULL;
> static Display *x11_display = NULL;
>
> static void
> _open_x11 ()
> {
> if (!x11_display)
> x11_display = XOpenDisplay (NULL);
> }
>
> static gboolean
> sync_bus_call (GstBus *bus, GstMessage *msg, gpointer data)
> {
> switch (GST_MESSAGE_TYPE (msg)) {
> case GST_MESSAGE_NEED_CONTEXT:
> {
> const gchar *context_type;
> GstContext *context = NULL;
>
> gst_message_parse_context_type (msg, &context_type);
> g_print("got need context %s\n", context_type);
>
> if (g_strcmp0 (context_type,
> GST_GL_DISPLAY_CONTEXT_TYPE) == 0) {
>
> _open_x11 ();
>
> if (!gl_display)
> gl_display = GST_GL_DISPLAY
> (gst_gl_display_x11_new_with_display (x11_display));
>
> context = gst_context_new
> (GST_GL_DISPLAY_CONTEXT_TYPE, TRUE);
> gst_context_set_gl_display (context, gl_display);
>
> gst_element_set_context (GST_ELEMENT (msg->src), context);
> }
>
> /* Here you'd retrieve the GstGLContext like in [4] and
> [5] and do a
> * similar thing as above with the display but matching
> types/names
> * with the "gst.gl.app_context" case in [6] */
>
> if (context)
> gst_context_unref (context);
> break;
> }
> default:
> break;
> }
>
> return FALSE;
> }
>
> > Thanks!
> > Andres
> >
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20150904/82455a0e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20150904/82455a0e/attachment-0001.sig>
More information about the gstreamer-devel
mailing list