OpenGL Texture via GstGLUpload - So close!
Lasse Laursen
lasse at lasselaursen.com
Thu Jan 15 07:22:57 PST 2015
Hey clever peoples,
I feel like I am so close to finally getting through to the other side
with this problem, so I'm hoping someone can give me a few clues as to
what pieces of the puzzle are missing. I used to use gStreamer as a pure
'decoding' library using this simple pipeline:
uridecodebin uri=" + mPathToVideoFile + " ! videoconvert ! appsink
name=sink caps="video/x-raw,format=RGB,pixel-aspect-ratio=1/1";
With this pipeline, I could quite easily grab the frames arriving at the
sink and use them however I pleased. Mostly just throwing them on a
texture in OpenGL and displaying them various places. This approach is -
of course - quite horrible. I'm not making use of gstreamer as a
fully-fledged media pipeline. So to rectify that - and because I'd like
to run some simple shaders on the video data - I am now using this
(almost as) simple pipeline:
uridecodebin uri=" + mPathToVideoFile + " ! glshader name=shader
location=" + pathToShader + " vars=\"float silver = float( 0.8 ); \" !
appsink name=sink caps=\"video/x-raw,format=RGB,pixel-aspect-ratio=1/1\"";
I make sure to share my current OpenGL context with the glshader element
thusly:
mPipeline_ShaderPre = gst_bin_get_by_name( GST_BIN( mGstPipeline ),
"shader" );
HGLRC mainContext = ::wglGetCurrentContext();
g_object_set( mPipeline_ShaderPre, "other-context", mainContext, NULL );
Not sure if this is correct to be honest, but I didn't see any errors in
gstreamers logs. Anyway - fast forward to my callback function that
fires whenever a sample is ready at the sink I proceed to grab a Sample,
from which I grab a Buffer, from which I grab a Frame, from which I grab
a Texture ID, thusly:
GstSample* gstVideoSinkSample = gst_app_sink_pull_sample( GST_APP_SINK(
appsink ) );
GstBuffer* gstBuffer = gst_sample_get_buffer( gstVideoSinkSample );
if ( !gst_video_frame_map( &gstFrame, &gstInfo, gstBuffer,
static_cast<GstMapFlags>( GST_MAP_READ | ( GST_MAP_FLAG_LAST << 1 ) ) ) )
{
errorToConsole( true, "Failed to map video frame" );
}
GLuint textureId = *(guint *) gstFrame.data[0];
The result is the number 6. Which I felt was a plausible Texture ID
handle. But when I then use a lovely tool named imdebug, and presume
that the target for the texture is GL_TEXTURE_2D (which might be wrong),
it throws a fit, because it attempts to determine the width and height
of said texture via these calls:
glBindTexture(target, texture);
glGetTexLevelParameteriv(target, 0, GL_TEXTURE_WIDTH, &w);
glGetTexLevelParameteriv(target, 0, GL_TEXTURE_HEIGHT, &h);
So clearly something is wrong. Some place along this winding road, I've
misunderstood something or overlooked something. I'd much appreciate any
suggestions/clues!
I myself am wondering if a callback at the sink is the right place to
grab the texture ID, but I'm not sure how to get at it any earlier, and
if that's wise. I am also unsure of how long this texture Id is valid if
I want to render it on to a quad. I hope someone can shed a bit of light!
Regards,
Lasse
More information about the gstreamer-devel
mailing list