<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
On 17/01/15 22:47, Lasse Laursen wrote:<br>
<blockquote cite="mid:%3C54BA4BE6.2030108@lasselaursen.com%3E"
type="cite">
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
Hey again,<br>
<br>
I can almost taste success. I imagine it being sweet, like sugar.
Entirely unlike the bitter lemony sourness of 'not quite there' I
have in my mouth now. Help me see the error of my ways!<br>
<br>
Inspired by the code Matthew sent my way, I've cobbled together
the following object in C++ that is supposed to create a pipeline,
share the current OpenGL context with it, grab samples arriving at
the appsink and provide the texture ID's to the main thread for
usage.<br>
<br>
videoTester.h<br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://hastebin.com/sutoyuxoci.coffee">http://hastebin.com/sutoyuxoci.coffee</a><br>
<br>
videoTester.cpp<br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://hastebin.com/gohofababi.cpp">http://hastebin.com/gohofababi.cpp</a><br>
<br>
In the main application, the following happens:<br>
<br>
Cinder sets up all the win32 app stuff, including an OpenGL
context.<br>
VideoTester4 is instantiated with paths pointing to a webm video
and a passthrough fragment shader (no parameters).<br>
prepPipeline() is called.<br>
launchPipeline() is called.<br>
The generated Texture ID is inspected via imdebug (an awesome
texture debugging tool) via this call:<br>
imdebugTexImage( GL_TEXTURE_2D, mVidTester4->getTexId(),
GL_RGBA );<br>
</blockquote>
<br>
This looks like a solid approach.<br>
<br>
A couple of things to try/check.<br>
<br>
1. See what the value of gl_context = wglGetCurrentContext()
returns. make sure it is not NULL. If it is, make sure you're
calling that in the main thread (or wherever your gl context is
current).<br>
2. if you call wglMakeCurrent (0, 0) make sure you reactivate it
after you're done as cinder might assume it controls the gl context
and might not reactivate it by itself.<br>
3. The use of GST_GL_API_OPENGL3 might not match cinder's API
usage. Also, the wgl backend does not currently create GL3 contexts
yet or contain gl debugging support from the library like the glx
backend.<br>
4. In order to remove shader issues, replace glshader by
glcolorscale for testing.<br>
5. enable all the gl debugging with GST_DEBUG=gl*:7 and see if the
texture handles you get from the callback match up with the textures
printed out. It might also mention some things about the context
setup in the beginning that might be useful.<br>
6. the lack of locking on mTextureId :)<br>
<br>
<blockquote cite="mid:%3C54BA4BE6.2030108@lasselaursen.com%3E"
type="cite"> Note that I intentionally do *not* unref any of the
associated frame/sample/buffer objects, just to keep things
simple. However, all I seem to get when I use imdebug to inspect
the textures, are empty 0,0 sized textures that (not surprisingly)
contain nothing at all. I wondered if GL_TEXTURE_2D might be the
wrong texture target and have also tried GL_TEXTURE_RECTANGLE
without any luck. Is there any place I can read the texture target
from?<br>
<br>
Anyway, the pipeline does not report any errors along the way, and
the gstreamer log only contains a few (harmless?) warnings:<br>
<br>
0:00:00.683792474 1836 0CFBEE60 WARN gldisplay
gstgldisplay.c:169:gst_gl_display_new: Could not create display.
user specified (NULL) (platform: (NULL)), creating dummy<br>
</blockquote>
<br>
That's normal for win32.<br>
<br>
<blockquote cite="mid:%3C54BA4BE6.2030108@lasselaursen.com%3E"
type="cite">0:00:00.686256033 1836 0CFBEE60
WARN basesrc
gstbasesrc.c:3470:gst_base_src_start_complete:<source> pad
not activated yet<br>
0:00:00.686752850 1836 0CFBEE60 WARN basesrc
gstbasesrc.c:3470:gst_base_src_start_complete:<source> pad
not activated yet<br>
</blockquote>
<br>
These may also be normal.<br>
<br>
<blockquote cite="mid:%3C54BA4BE6.2030108@lasselaursen.com%3E"
type="cite"> The texture ID increases as the program continues
execution (which is to be expected), and inspecting the Textures
in the actual callback thread with imdebug crashes the program.
This has to be due to the fact that the callback function executes
in a separate thread as Matthew told me last we spoke.<br>
</blockquote>
<br>
My guess based on this and the above is something simple based on
the setup of the gl contexts is off.<br>
<br>
<blockquote cite="mid:%3C54BA4BE6.2030108@lasselaursen.com%3E"
type="cite"> I've poked a bit around with the various Gst objects
without any luck so far. Any ideas/clues as to what I'm missing or
how to narrow down what the problem might be?<br>
<br>
Regards,<br>
Lasse<br>
<br>
<div class="moz-cite-prefix">On 16-01-2015 22:21, Matthew Waters
wrote:<br>
</div>
<blockquote cite="mid:54B98EEE.1020106@gmail.com" type="cite">
<meta content="text/html; charset=utf-8"
http-equiv="Content-Type">
<br>
<div class="moz-cite-prefix">On 17/01/15 06:42, Lasse Laursen
wrote:<br>
</div>
<blockquote cite="mid:%3C54B96992.1030209@lasselaursen.com%3E"
type="cite">
<meta content="text/html; charset=utf-8"
http-equiv="Content-Type">
Hey Matthew,<br>
<br>
Thanks for the reply and especially the example. I'm working
on getting it function with the cinder framework. At first it
seemed quite odd to hand over the context as a result of a bus
message, but I can see the logic in not needing to create a
secondary OpenGL context for gstreamer to make textures in, if
one already exists. I do have one question though:<br>
<br>
Is it vital to turn off the OpenGL context when sharing it
with Gstreamer? I've done a little context sharing before and
I don't recall having to actively turn any contexts 'off'.<br>
</blockquote>
<br>
No it is not required to turn off the context.<br>
<br>
<blockquote cite="mid:%3C54B96992.1030209@lasselaursen.com%3E"
type="cite"> Regards,<br>
Lasse<br>
<br>
<div class="moz-cite-prefix">On 16-01-2015 00:27, Matthew
Waters wrote:<br>
</div>
<blockquote cite="mid:54B85AE6.6000607@gmail.com" type="cite">
<meta content="text/html; charset=utf-8"
http-equiv="Content-Type">
A couple of points added inline.<br>
<br>
<div class="moz-cite-prefix">On 16/01/15 03:14, Lasse
Laursen wrote:<br>
</div>
<blockquote
cite="mid:%3C54B7E76D.6000904@lasselaursen.com%3E"
type="cite">
<meta http-equiv="content-type" content="text/html;
charset=utf-8">
Hello!<br>
<br>
Apologies for the double post, but my inept handling of
Thunderbird caused my last post to be appended to an
unrelated thread. Sorry about that Luis dArquer.<br>
<br>
For the sake of posterity, here's is the original
question:<br>
<br>
I feel like I am so close to finally getting through to
the other side with this problem, so I'm hoping someone
can give me a few clues as to what pieces of the puzzle
are missing. I used to use gStreamer as a pure 'decoding'
library using this simple pipeline: <br>
<br>
uridecodebin uri=" + mPathToVideoFile + " !
videoconvert ! appsink name=sink
caps="video/x-raw,format=RGB,pixel-aspect-ratio=1/1"; <br>
<br>
With this pipeline, I could quite easily grab the frames
arriving at the sink and use them however I pleased.
Mostly just throwing them on a texture in OpenGL and
displaying them various places. This approach is - of
course - quite horrible. I'm not making use of gstreamer
as a fully-fledged media pipeline. So to rectify that -
and because I'd like to run some simple shaders on the
video data - I am now using this (almost as) simple
pipeline: <br>
<br>
uridecodebin uri=" + mPathToVideoFile + " ! glshader
name=shader location=" + pathToShader + " vars=\"float
silver = float( 0.8 ); \" ! appsink name=sink
caps=\"video/x-raw,format=RGB,pixel-aspect-ratio=1/1\""; <br>
</blockquote>
<br>
1. You should set the appsink caps to contain the
memory:GstGLMemory caps features. e.g.
video/x-raw(memory:GstGLMemory),format=RGBA... see
gst-inspect-1.0 glshader for the supported formats and
features. Otherwise, glshader (and any GL element) will
download the texture upon seeing the system memory caps.<br>
<br>
<blockquote
cite="mid:%3C54B7E76D.6000904@lasselaursen.com%3E"
type="cite"> I make sure to share my current OpenGL
context with the glshader element thusly: <br>
<br>
mPipeline_ShaderPre = gst_bin_get_by_name( GST_BIN(
mGstPipeline ), "shader" ); <br>
HGLRC mainContext = ::wglGetCurrentContext(); <br>
g_object_set( mPipeline_ShaderPre, "other-context",
mainContext, NULL ); <br>
</blockquote>
<br>
2. The "other-context" property expects a GstGLContext
rather than a platform specific handle. You can wrap a
context handle using gst_gl_context_new_wrapped(). However,
in master there is no "other-context" property anymore so
the sharing of GL context happens using GstContext. See for
example <a moz-do-not-send="true"
href="http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tests/examples/gl/sdl/sdlshare.c#n232">http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tests/examples/gl/sdl/sdlshare.c#n232</a>
and the GstContext documentation.<br>
<br>
<blockquote
cite="mid:%3C54B7E76D.6000904@lasselaursen.com%3E"
type="cite"> Not sure if this is correct to be honest, but
I didn't see any errors in gstreamers logs. Anyway - fast
forward to my callback function that fires whenever a
sample is ready at the sink I proceed to grab a Sample,
from which I grab a Buffer, from which I grab a Frame,
from which I grab a Texture ID, thusly: <br>
<br>
GstSample* gstVideoSinkSample = gst_app_sink_pull_sample(
GST_APP_SINK( appsink ) ); <br>
GstBuffer* gstBuffer = gst_sample_get_buffer(
gstVideoSinkSample ); <br>
if ( !gst_video_frame_map( &gstFrame, &gstInfo,
gstBuffer, static_cast<GstMapFlags>( GST_MAP_READ |
( GST_MAP_FLAG_LAST << 1 ) ) ) ) <br>
{ <br>
errorToConsole( true, "Failed to map video frame"
); <br>
} <br>
GLuint textureId = *(guint *) gstFrame.data[0]; <br>
<br>
The result is the number 6. Which I felt was a plausible
Texture ID handle. But when I then use a lovely tool named
imdebug, and presume that the target for the texture is
GL_TEXTURE_2D (which might be wrong), it throws a fit,
because it attempts to determine the width and height of
said texture via these calls: <br>
<br>
glBindTexture(target, texture); <br>
glGetTexLevelParameteriv(target, 0, GL_TEXTURE_WIDTH,
&w); <br>
glGetTexLevelParameteriv(target, 0, GL_TEXTURE_HEIGHT,
&h); <br>
<br>
So clearly something is wrong. Some place along this
winding road, I've misunderstood something or overlooked
something. I'd much appreciate any suggestions/clues! <br>
<br>
I myself am wondering if a callback at the sink is the
right place to grab the texture ID, but I'm not sure how
to get at it any earlier, and if that's wise. I am also
unsure of how long this texture Id is valid if I want to
render it on to a quad. I hope someone can shed a bit of
light! <br>
</blockquote>
<br>
It will be valid for as long as it's not overwritten/reused
by the producing element which generally means while you
keep a ref to the buffer or have the video frame mapped.<br>
<br>
<blockquote
cite="mid:%3C54B7E76D.6000904@lasselaursen.com%3E"
type="cite"> Regards, <br>
Lasse <br>
<br>
<div class="moz-signature">-- <br>
Lasse Farnung Laursen<br>
Researcher at the University of Tokyo<br>
<a moz-do-not-send="true"
href="http://www.lasselaursen.com">www.lasselaursen.com</a><br>
Twitter: <a moz-do-not-send="true"
href="https://twitter.com/PMP_J">@PMP_J</a></div>
</blockquote>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
gstreamer-devel mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:gstreamer-devel@lists.freedesktop.org">gstreamer-devel@lists.freedesktop.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel">http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel</a>
</pre>
</blockquote>
<br>
<div class="moz-signature">-- <br>
Lasse Farnung Laursen<br>
Researcher at the University of Tokyo<br>
<a moz-do-not-send="true" href="http://www.lasselaursen.com">www.lasselaursen.com</a><br>
Twitter: <a moz-do-not-send="true"
href="https://twitter.com/PMP_J">@PMP_J</a></div>
</blockquote>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
gstreamer-devel mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:gstreamer-devel@lists.freedesktop.org">gstreamer-devel@lists.freedesktop.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel">http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel</a>
</pre>
</blockquote>
<br>
<div class="moz-signature">-- <br>
Lasse Farnung Laursen<br>
Researcher at the University of Tokyo<br>
<a moz-do-not-send="true" href="http://www.lasselaursen.com">www.lasselaursen.com</a><br>
Twitter: <a moz-do-not-send="true"
href="https://twitter.com/PMP_J">@PMP_J</a></div>
</blockquote>
<br>
</body>
</html>