OpenGL Texture via GstGLUpload - So close! - Proper Post

Fri Jan 16 14:21:34 PST 2015

On 17/01/15 06:42, Lasse Laursen wrote:
> Hey Matthew,
>
> Thanks for the reply and especially the example. I'm working on
> getting it function with the cinder framework. At first it seemed
> quite odd to hand over the context as a result of a bus message, but I
> can see the logic in not needing to create a secondary OpenGL context
> for gstreamer to make textures in, if one already exists. I do have
> one question though:
>
> Is it vital to turn off the OpenGL context when sharing it with
> Gstreamer? I've done a little context sharing before and I don't
> recall having to actively turn any contexts 'off'.

No it is not required to turn off the context.

> Regards,
> Lasse
>
> On 16-01-2015 00:27, Matthew Waters wrote:
>> A couple of points added inline.
>>
>> On 16/01/15 03:14, Lasse Laursen wrote:
>>> Hello!
>>>
>>> Apologies for the double post, but my inept handling of Thunderbird
>>> caused my last post to be appended to an unrelated thread. Sorry
>>> about that Luis dArquer.
>>>
>>> For the sake of posterity, here's is the original question:
>>>
>>> I feel like I am so close to finally getting through to the other
>>> side with this problem, so I'm hoping someone can give me a few
>>> clues as to what pieces of the puzzle are missing. I used to use
>>> gStreamer as a pure 'decoding' library using this simple pipeline:
>>>
>>> uridecodebin uri=" + mPathToVideoFile +    " ! videoconvert !
>>> appsink name=sink caps="video/x-raw,format=RGB,pixel-aspect-ratio=1/1";
>>>
>>> With this pipeline, I could quite easily grab the frames arriving at
>>> the sink and use them however I pleased. Mostly just throwing them
>>> on a texture in OpenGL and displaying them various places. This
>>> approach is - of course - quite horrible. I'm not making use of
>>> gstreamer as a fully-fledged media pipeline. So to rectify that -
>>> and because I'd like to run some simple shaders on the video data -
>>> I am now using this (almost as) simple pipeline:
>>>
>>> uridecodebin uri=" + mPathToVideoFile + " ! glshader name=shader
>>> location=" + pathToShader + " vars=\"float silver = float( 0.8 ); \"
>>> ! appsink name=sink
>>> caps=\"video/x-raw,format=RGB,pixel-aspect-ratio=1/1\"";
>>
>> 1. You should set the appsink caps to contain the memory:GstGLMemory
>> caps features. e.g. video/x-raw(memory:GstGLMemory),format=RGBA...
>> see gst-inspect-1.0 glshader for the supported formats and features. 
>> Otherwise, glshader (and any GL element) will download the texture
>> upon seeing the system memory caps.
>>
>>> I make sure to share my current OpenGL context with the glshader
>>> element thusly:
>>>
>>> mPipeline_ShaderPre = gst_bin_get_by_name( GST_BIN( mGstPipeline ),
>>> "shader" );
>>> HGLRC mainContext = ::wglGetCurrentContext();
>>> g_object_set( mPipeline_ShaderPre, "other-context", mainContext,
>>> NULL );
>>
>> 2. The "other-context" property expects a GstGLContext rather than a
>> platform specific handle.  You can wrap a context handle using
>> gst_gl_context_new_wrapped().  However, in master there is no
>> "other-context" property anymore so the sharing of GL context happens
>> using GstContext.  See for example
>> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tests/examples/gl/sdl/sdlshare.c#n232
>> and the GstContext documentation.
>>
>>> Not sure if this is correct to be honest, but I didn't see any
>>> errors in gstreamers logs. Anyway - fast forward to my callback
>>> function that fires whenever a sample is ready at the sink I proceed
>>> to grab a Sample, from which I grab a Buffer, from which I grab a
>>> Frame, from which I grab a Texture ID, thusly:
>>>
>>> GstSample* gstVideoSinkSample = gst_app_sink_pull_sample(
>>> GST_APP_SINK( appsink ) );
>>> GstBuffer* gstBuffer = gst_sample_get_buffer( gstVideoSinkSample );
>>> if ( !gst_video_frame_map( &gstFrame, &gstInfo, gstBuffer,
>>> static_cast<GstMapFlags>( GST_MAP_READ | ( GST_MAP_FLAG_LAST << 1 )
>>> ) ) )
>>>     {
>>>         errorToConsole( true, "Failed to map video frame" );
>>>     }
>>> GLuint textureId = *(guint *) gstFrame.data[0];
>>>
>>> The result is the number 6. Which I felt was a plausible Texture ID
>>> handle. But when I then use a lovely tool named imdebug, and presume
>>> that the target for the texture is GL_TEXTURE_2D (which might be
>>> wrong), it throws a fit, because it attempts to determine the width
>>> and height of said texture via these calls:
>>>
>>> glBindTexture(target, texture);
>>>   glGetTexLevelParameteriv(target, 0, GL_TEXTURE_WIDTH, &w);
>>>   glGetTexLevelParameteriv(target, 0, GL_TEXTURE_HEIGHT, &h);
>>>
>>> So clearly something is wrong. Some place along this winding road,
>>> I've misunderstood something or overlooked something. I'd much
>>> appreciate any suggestions/clues!
>>>
>>> I myself am wondering if a callback at the sink is the right place
>>> to grab the texture ID, but I'm not sure how to get at it any
>>> earlier, and if that's wise. I am also unsure of how long this
>>> texture Id is valid if I want to render it on to a quad. I hope
>>> someone can shed a bit of light!
>>
>> It will be valid for as long as it's not overwritten/reused by the
>> producing element which generally means while you keep a ref to the
>> buffer or have the video frame mapped.
>>
>>> Regards,
>>> Lasse
>>>
>>> -- 
>>> Lasse Farnung Laursen
>>> Researcher at the University of Tokyo
>>> www.lasselaursen.com <http://www.lasselaursen.com>
>>> Twitter: @PMP_J <https://twitter.com/PMP_J>
>>
>>
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> -- 
> Lasse Farnung Laursen
> Researcher at the University of Tokyo
> www.lasselaursen.com <http://www.lasselaursen.com>
> Twitter: @PMP_J <https://twitter.com/PMP_J>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20150117/cf484bb0/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20150117/cf484bb0/attachment-0001.sig>