OpenGL Texture via GstGLUpload - So close! - Proper Post

[Matthew Waters]

A couple of points added inline.

On 16/01/15 03:14, Lasse Laursen wrote:
> Hello!
>
> Apologies for the double post, but my inept handling of Thunderbird
> caused my last post to be appended to an unrelated thread. Sorry about
> that Luis dArquer.
>
> For the sake of posterity, here's is the original question:
>
> I feel like I am so close to finally getting through to the other side
> with this problem, so I'm hoping someone can give me a few clues as to
> what pieces of the puzzle are missing. I used to use gStreamer as a
> pure 'decoding' library using this simple pipeline:
>
> uridecodebin uri=" + mPathToVideoFile +    " ! videoconvert ! appsink
> name=sink caps="video/x-raw,format=RGB,pixel-aspect-ratio=1/1";
>
> With this pipeline, I could quite easily grab the frames arriving at
> the sink and use them however I pleased. Mostly just throwing them on
> a texture in OpenGL and displaying them various places. This approach
> is - of course - quite horrible. I'm not making use of gstreamer as a
> fully-fledged media pipeline. So to rectify that - and because I'd
> like to run some simple shaders on the video data - I am now using
> this (almost as) simple pipeline:
>
> uridecodebin uri=" + mPathToVideoFile + " ! glshader name=shader
> location=" + pathToShader + " vars=\"float silver = float( 0.8 ); \" !
> appsink name=sink
> caps=\"video/x-raw,format=RGB,pixel-aspect-ratio=1/1\"";

1. You should set the appsink caps to contain the memory:GstGLMemory
caps features. e.g. video/x-raw(memory:GstGLMemory),format=RGBA... see
gst-inspect-1.0 glshader for the supported formats and features. 
Otherwise, glshader (and any GL element) will download the texture upon
seeing the system memory caps.

> I make sure to share my current OpenGL context with the glshader
> element thusly:
>
> mPipeline_ShaderPre = gst_bin_get_by_name( GST_BIN( mGstPipeline ),
> "shader" );
> HGLRC mainContext = ::wglGetCurrentContext();
> g_object_set( mPipeline_ShaderPre, "other-context", mainContext, NULL );

2. The "other-context" property expects a GstGLContext rather than a
platform specific handle.  You can wrap a context handle using
gst_gl_context_new_wrapped().  However, in master there is no
"other-context" property anymore so the sharing of GL context happens
using GstContext.  See for example
http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tests/examples/gl/sdl/sdlshare.c#n232
and the GstContext documentation.

> Not sure if this is correct to be honest, but I didn't see any errors
> in gstreamers logs. Anyway - fast forward to my callback function that
> fires whenever a sample is ready at the sink I proceed to grab a
> Sample, from which I grab a Buffer, from which I grab a Frame, from
> which I grab a Texture ID, thusly:
>
> GstSample* gstVideoSinkSample = gst_app_sink_pull_sample(
> GST_APP_SINK( appsink ) );
> GstBuffer* gstBuffer = gst_sample_get_buffer( gstVideoSinkSample );
> if ( !gst_video_frame_map( &gstFrame, &gstInfo, gstBuffer,
> static_cast<GstMapFlags>( GST_MAP_READ | ( GST_MAP_FLAG_LAST << 1 ) ) ) )
>     {
>         errorToConsole( true, "Failed to map video frame" );
>     }
> GLuint textureId = *(guint *) gstFrame.data[0];
>
> The result is the number 6. Which I felt was a plausible Texture ID
> handle. But when I then use a lovely tool named imdebug, and presume
> that the target for the texture is GL_TEXTURE_2D (which might be
> wrong), it throws a fit, because it attempts to determine the width
> and height of said texture via these calls:
>
> glBindTexture(target, texture);
>   glGetTexLevelParameteriv(target, 0, GL_TEXTURE_WIDTH, &w);
>   glGetTexLevelParameteriv(target, 0, GL_TEXTURE_HEIGHT, &h);
>
> So clearly something is wrong. Some place along this winding road,
> I've misunderstood something or overlooked something. I'd much
> appreciate any suggestions/clues!
>
> I myself am wondering if a callback at the sink is the right place to
> grab the texture ID, but I'm not sure how to get at it any earlier,
> and if that's wise. I am also unsure of how long this texture Id is
> valid if I want to render it on to a quad. I hope someone can shed a
> bit of light!

It will be valid for as long as it's not overwritten/reused by the
producing element which generally means while you keep a ref to the
buffer or have the video frame mapped.

> Regards,
> Lasse
>
> -- 
> Lasse Farnung Laursen
> Researcher at the University of Tokyo
> www.lasselaursen.com <http://www.lasselaursen.com>
> Twitter: @PMP_J <https://twitter.com/PMP_J>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20150116/96d096d5/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20150116/96d096d5/attachment.sig>