<div dir="ltr">Thanks Matthew,<div>to confirm it i did a little patch to check if the passed GstMemory was indeed a GstGLMemoryEGL, but apparently it is not, the below assert failed right after the qt video sink received a new frame.</div><div><br></div><div>GstMemory *mem = gst_buffer_peek_memory (buffer, 0);<br></div><div>g_assert (gst_is_gl_memory_egl (mem));</div><div><br></div><div>As you suggested, I tried to execute the pipeline 'omxh264dec ! glimagesinkelement', but it ends up giving me an error when calling glGetString(GL_VERSION).</div><div><br></div><div>$ gst-launch-1.0 filesrc location=/home/root/test.mp4 ! omxh264dec ! glimagesinkelement<br></div><div><br></div><div> glcontext gstglcontext_egl.c:464:gst_gl_context_egl_choose_config:<glcontextegl0> chosen EGLConfig <br> glcontext gstglcontext_egl.c:174:gst_gl_context_egl_dump_config:<glcontextegl0> dumping EGLConfig 0x4 with id 0x4 and native visual id 0x8428 of type 0x3038 <br> glcontext gstglcontext_egl.c:207:gst_gl_context_egl_dump_config:<glcontextegl0> Conformant for OpenGL ES|OpenGL ES 2.x|OpenVG <br> glcontext gstglcontext_egl.c:240:gst_gl_context_egl_dump_config:<glcontextegl0> Renderable for OpenGL ES|OpenGL ES 2.x|OpenVG <br> glcontext gstglcontext_egl.c:273:gst_gl_context_egl_dump_config:<glcontextegl0> Surface for window|pbuffer|multisample-resolve-box|swap-behaviour-preserved|vg-alpha-format-pre|vg-colorspace-linear<br> glcontext gstglcontext_egl.c:313:gst_gl_context_egl_dump_config:<glcontextegl0> [R, G, B, A] = [8, 8, 8, 0] <br> glcontext gstglcontext_egl.c:335:gst_gl_context_egl_dump_config:<glcontextegl0> [D, S] = [24, 0] <br> glcontext gstglcontext_egl.c:346:gst_gl_context_egl_dump_config:<glcontextegl0> Swap interval range is [0, 2147483647] <br> glcontext gstglcontext_egl.c:360:gst_gl_context_egl_dump_config:<glcontextegl0> PBuffer maximum dimensions are [2048, 2048]. Max pixels are 4194304 <br> glcontext gstglcontext_egl.c:373:gst_gl_context_egl_dump_config:<glcontextegl0> Multisample buffers: 0 and Samples per pixel: 0 <br> glcontext gstglcontext_egl.c:501:_create_context_with_flags:<glcontextegl0> attempting to create OpenGL ES context version 2.0 flags 0 profile 0 <br> glcontext gstglcontext_egl.c:746:gst_gl_context_egl_create_context: gl context created: 1 <br> glwindow gstglwindow_dispmanx_egl.c:226:window_resize: resizing invisible window from 0x0 to 16x16 <br> videosink gstvideosink.c:132:gst_video_sink_center_rect: source is 16x16 dest is 640x480, result is 16x16 with x,y 312x232 <br> glcontext gstglcontext_egl.c:794:gst_gl_context_egl_create_context: Creating EGLSurface from window_handle 0x5fd100 <br> glcontext gstglcontext_egl.c:828:gst_gl_context_egl_create_context: surface created <br> glcontext gstglcontext.c:1247:gst_gl_context_create_thread:<glcontextegl0> created context <br> glcontext gstglcontext.c:750:gst_gl_context_activate:<glcontextegl0> activate:1 <br> glcontext gstglcontext.c:1263:gst_gl_context_create_thread:<glcontextegl0> available GL APIs: gles2 <br> glcontext gstglcontext.c:1282:gst_gl_context_create_thread:<glcontextegl0> Filling info <br> glcontext gstglcontext.c:1042:gst_gl_context_create:<glcontextegl0> gl thread created <br> glimagesink gstglimagesink.c:1010:_ensure_gl_setup:<glimagesink0> <b>error: glGetString not defined or returned invalid value</b><br></div><div><br></div><div>After some searching I find out that this problem could be related with the wrong GL lib loading. So I certified that gst-launch-1.0 was loading the correct glGetString version.</div><div><br></div><div>My system listed two libraries with the symbol:</div><div><br></div>$ find . -name \*.so -exec bash -c "nm --defined-only -D {} 2>/dev/null | grep glGetString && echo {}" \;<br>00008810 T glGetString<br>./<b>libbrcmGLESv2.so</b><br>0000880c T glGetString<br><div>./<b>libGLESv2.so</b></div><div> </div><div>According to rpi doc, the corrected one is the brcm. Thus, I ran gdb to check which one is being loaded right before glGetString(GL_VERSION) was called. gst_gl_display_create_context was chosen to the the point of verification.</div><div> </div><div>$ gdb gst-launch-1.0</div><div>Thread 1 "gst-launch-1.0" hit Breakpoint 2, 0x7693fa22 in gst_gl_display_create_context () from /usr/lib/libgstgl-1.0.so.0<br>(gdb) info sharedlibrary <br>From To Syms Read Shared Object Library<br>0x76fd5b80 0x76fe96b0 Yes (*) /lib/ld-linux-armhf.so.3<br>0x76f0d9d8 0x76f94684 Yes (*) /usr/lib/libgstreamer-1.0.so.0<br>0x76e1d7e8 0x76e7850c Yes (*) /usr/lib/libglib-2.0.so.0<br>0x76dc2c40 0x76de9810 Yes (*) /usr/lib/libgobject-2.0.so.0<br>0x76d99540 0x76da4b94 Yes (*) /lib/libpthread.so.0<br>0x76cbba40 0x76d61ac4 Yes (*) /lib/libc.so.6<br>0x76c8dbb0 0x76c8e6dc Yes (*) /usr/lib/libgmodule-2.0.so.0<br>0x76c32148 0x76c59c60 Yes (*) /lib/libm.so.6<br>0x76c16b38 0x76c17518 Yes (*) /lib/libdl.so.2<br>0x76bb8f48 0x76befd8c Yes (*) /usr/lib/libpcre.so.1<br>0x76ba3360 0x76ba6680 Yes (*) /usr/lib/libffi.so.7<br>0x76b85118 0x76b8ff10 Yes (*) /lib/libgcc_s.so.1<br>0x76b28520 0x76b59774 Yes (*) /usr/lib/gstreamer-1.0/libgstcoreelements.so<br>0x76ac55e8 0x76b012a4 Yes (*) /usr/lib/libgstbase-1.0.so.0<br>0x76a77e08 0x76a9ce4c Yes (*) /usr/lib/gstreamer-1.0/libgstomx.so<br>0x769fedd0 0x76a4623c Yes (*) /usr/lib/libgstvideo-1.0.so.0<br>0x76998058 0x769ce23c Yes (*) /usr/lib/libgstaudio-1.0.so.0<br>0x7697ad60 0x7697b910 Yes (*) /usr/lib/libgstallocators-1.0.so.0<br>0x769355a0 0x76957ae8 Yes (*) /usr/lib/libgstgl-1.0.so.0<br>0x768ccfe8 0x7690893c Yes (*) /usr/lib/liborc-0.4.so.0<br>0x76892618 0x768a9e14 Yes (*) /usr/lib/libgsttag-1.0.so.0<br>0x76878a0c 0x76879ed4 Yes (*) /usr/lib/libbcm_host.so<br>0x7685114c 0x768642dc Yes (*) <b>/usr/lib/libbrcmEGL.so</b><br>0x76829928 0x76831dc4 Yes (*) /usr/lib/libvchostif.so<br>0x76806dc8 0x76812f2c Yes (*) <b>/usr/lib/libbrcmGLESv2.so</b><br>0x767e3bc8 0x767ecc88 Yes (*) /lib/libz.so.1<br>0x767cd704 0x767cfb30 Yes (*) /usr/lib/libvchiq_arm.so<br>0x767b6c4c 0x767b9e28 Yes (*) /usr/lib/libvcos.so<br>0x7679fae0 0x767a1f30 Yes (*) /lib/librt.so.1<br>0x76764228 0x7677d648 Yes (*) /usr/lib/gstreamer-1.0/libgstopengl.so<br>0x76742240 0x76748fb4 Yes (*) /usr/lib/libgstcontroller-1.0.so.0<br>0x767135b0 0x7672a9f0 Yes (*) /usr/lib/libpng16.so.16<br>0x766d89e0 0x766fb670 Yes (*) /usr/lib/libjpeg.so.62<br></div><div><br></div><div>It's picking the right lib (libbrcmGLESv2.so) so glGetString is returning a invalid value.</div><div>Finally, to validate i coded a simple sample to call glGetString, which surprisingly returned : <b>OpenGL ES 2.0</b></div><div><br></div><div>What could be wrong with glimagesinkelement ?</div><div><br></div><div>regards. <br><div><table cellpadding="0" class="gmail-cf gmail-gJ" style="border-collapse:collapse;margin-top:0px;width:auto;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:14px;letter-spacing:0.2px;display:block"></table></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Jun 21, 2020 at 11:39 AM Matthew Waters <<a href="mailto:ystreet00@gmail.com">ystreet00@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div>The OpenGL texture out of omxh264dec on
the RPi is wrapped in a GstGLMemoryEGL which already wraps the
EGLImage produced by the OMX decoder. I'm not entirely sure what
VCSM is or how it actually relates to all this but I assume that
the following pipeline works for you? 'omxh264dec !
glimagesinkelement' If so, that is already using the
EGLImage/OpenGL texture produced by omxh264dec and rendering using
OpenGL with glimagesink. It is very hard in general for the RPi
to do anything but only decode and display a 1080p@30 video and
even then it can just barely do it.<br>
<br>
On 21/6/20 10:22 pm, Rafael Savignon wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="auto">Hi all,
<div dir="auto">I'm developing a Qt5 application to playback
hardware decoded h.264 files. But on raspberry pi 3 its
struggling to play media greater than 720p. After some
investigation of <b>QGstVideoBuffer</b> class I realized
that calls to gst_video_frame_map function passing bigger
frames were consuming much cpu time ~50ms, which was
resulting in the postponing of next frames delivery. So I
had to find out a more performatic way to read the video
frame content. After some search I found out that a special
buffer mechanism could enable applications to map GPU memory
directly to addressable process memory, avoiding unnecessary
copy (VCSM). Then I took another look at QtMultimedia source
code and saw that a OpenGL texture handle was being packaged
in with <b>QGstVideoBuffer</b>, as shown below.</div>
<div dir="auto"><span style="background-color:rgba(0,0,0,0.024);color:rgb(46,139,87);font-family:monaco,"andale mono","courier new",courier,mono;font-size:12.1007px;white-space:pre-wrap">
</span></div>
<div dir="auto">guint <b>textureId</b> =
gst_gl_memory_get_texture_id(glmem);<br>
</div>
<div dir="auto">videoBuffer = new QGstVideoBuffer(buffer,
m_videoInfo, m_format.handleType(), <b>textureId</b>); <b style="font-family:sans-serif"><br>
</b></div>
<div dir="auto"><br>
</div>
<div dir="auto">So I thought that things were starting to get
clear. Thus I decided to use vcsm to map the texture and
speed things up .</div>
<div dir="auto"><br>
</div>
<div dir="auto">First, on my <b>QAbstractVideoSurface::start</b>
implemented method I initialized vcsm and created a
EGLimageKHR.</div>
<div dir="auto"><br>
</div>
<div dir="auto">int w = Util::nextPOT(size.width());</div>
<div dir="auto">int h = Util::nextPOT(size.height());</div>
<div dir="auto">const EGLint attrib[] = {
EGL_IMAGE_PRESERVED_KHR, EGL_TRUE, EGL_NONE, EGL_NONE
};<br>
</div>
<div dir="auto">vcsm_info.width = w;<br>
</div>
<div dir="auto">
<div dir="auto">vcsm_info.height = h;</div>
<br>
</div>
<div dir="auto">vcsm_init();</div>
<div dir="auto">eglFbImage =
eglCreateImageKHR(eglGetCurrentDisplay(), EGL_NO_CONTEXT,
EGL_IMAGE_BRCM_VCSM, &vcsm_info, attrib); </div>
<div dir="auto"><br>
</div>
<div dir="auto">Second, on my <b>QAbstractVideoSurface::present</b>
implementation method I tried to grab the passed Texture
content, but it didn't work. Despite this, all OpenGL calls
succeeded. I can access the mapped buffer, but it doesn't
contain anything meaningful.<br>
</div>
<div dir="auto"><br>
</div>
<div dir="auto">QOpenGLFunctions* f = ctx->functions(); </div>
<div dir="auto">GLuint framebuffer; </div>
<div dir="auto">GLuint depthRenderbuffer; </div>
<div dir="auto">GLint prevFbo;</div>
<div dir="auto">GLenum status = GL_FRAMEBUFFER_COMPLETE;</div>
<div dir="auto">GLuint texture = static_cast<GLuint>(
currentFrame.handle().toInt() );</div>
<div dir="auto">int texWidth =
Util::nextPOT(currentFrame.width());</div>
<div dir="auto">int texHeight =
Util::nextPOT(currentFrame.height());</div>
<div dir="auto"> </div>
<div dir="auto">GLCHK(f->glGetIntegerv(
GL_FRAMEBUFFER_BINDING, &prevFbo )); </div>
<div dir="auto">GLCHK(f->glGenFramebuffers(1,
&framebuffer));</div>
<div dir="auto">GLCHK(f->glBindFramebuffer(GL_FRAMEBUFFER,
framebuffer));</div>
<div dir="auto">GLCHK(glActiveTexture(GL_TEXTURE0));</div>
<div dir="auto">GLCHK(glBindTexture(GL_TEXTURE_2D, texture));</div>
<div dir="auto">GLCHK(glTexParameteri(GL_TEXTURE_2D,
GL_TEXTURE_MAG_FILTER, GL_NEAREST));</div>
<div dir="auto">GLCHK(glTexParameteri(GL_TEXTURE_2D,
GL_TEXTURE_MIN_FILTER, GL_NEAREST)); </div>
<div dir="auto">GLCHK(glEGLImageTargetTexture2DOES(GL_TEXTURE_2D,
eglFbImage)); </div>
<div dir="auto">GLCHK(f->glFramebufferTexture2D(GL_FRAMEBUFFER,
GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, texture, 0)); </div>
<div dir="auto">GLCHK(glBindTexture(GL_TEXTURE_2D, 0)); </div>
<div dir="auto">status =
glCheckFramebufferStatus(GL_FRAMEBUFFER);</div>
<div dir="auto">// check fbo status </div>
<div dir="auto">GLCHK(f->glFinish()); </div>
<div dir="auto"><br>
</div>
<div dir="auto">uint8_t *vcsmBuffer; </div>
<div dir="auto">VCSM_CACHE_TYPE_T cacheType; </div>
<div dir="auto">vcsmBuffer =
(uint8_t*)vcsm_lock_cache(vcsm_info.vcsm_handle,
VCSM_CACHE_TYPE_HOST, &cacheType);</div>
<div dir="auto">// print buffer</div>
<div dir="auto">vcsm_unlock_ptr(vcsmBuffer); </div>
<div dir="auto"><br>
</div>
<div dir="auto">Is the Texture Id correctly filled by
omxh264dec or glupload ?</div>
<div dir="auto">Can i bind the texture id to a different fbo ?</div>
<div dir="auto">Why <b style="font-family:sans-serif">gst_video_frame_map
</b>correctly map the video frame content and vcsm do not
despite being addressable?</div>
<div dir="auto"><b style="font-family:sans-serif"><br>
</b></div>
<div dir="auto">If someone could give a tip on this I would be
thankful.<br style="border:none;margin:0px;padding:0px">
</div>
<div dir="auto"><br>
</div>
<div>regards.</div>
<div dir="auto"><br>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
gstreamer-devel mailing list
<a href="mailto:gstreamer-devel@lists.freedesktop.org" target="_blank">gstreamer-devel@lists.freedesktop.org</a>
<a href="https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel" target="_blank">https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel</a>
</pre>
</blockquote>
<br>
</div>
</blockquote></div>