<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">The OpenGL texture out of omxh264dec on
the RPi is wrapped in a GstGLMemoryEGL which already wraps the
EGLImage produced by the OMX decoder. I'm not entirely sure what
VCSM is or how it actually relates to all this but I assume that
the following pipeline works for you? 'omxh264dec !
glimagesinkelement' If so, that is already using the
EGLImage/OpenGL texture produced by omxh264dec and rendering using
OpenGL with glimagesink. It is very hard in general for the RPi
to do anything but only decode and display a 1080p@30 video and
even then it can just barely do it.<br>
<br>
On 21/6/20 10:22 pm, Rafael Savignon wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAEXJbNSdaPrPJh8CPqkJvSFwqDqCwxdzDDTDXhZSK8QdU2baSQ@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div dir="auto">Hi all,
<div dir="auto">I'm developing a Qt5 application to playback
hardware decoded h.264 files. But on raspberry pi 3 its
struggling to play media greater than 720p. After some
investigation of <b>QGstVideoBuffer</b> class I realized
that calls to gst_video_frame_map function passing bigger
frames were consuming much cpu time ~50ms, which was
resulting in the postponing of next frames delivery. So I
had to find out a more performatic way to read the video
frame content. After some search I found out that a special
buffer mechanism could enable applications to map GPU memory
directly to addressable process memory, avoiding unnecessary
copy (VCSM). Then I took another look at QtMultimedia source
code and saw that a OpenGL texture handle was being packaged
in with <b>QGstVideoBuffer</b>, as shown below.</div>
<div dir="auto"><span style="background-color:rgba(0,0,0,0.024);color:rgb(46,139,87);font-family:monaco,"andale mono","courier new",courier,mono;font-size:12.1007px;white-space:pre-wrap">
</span></div>
<div dir="auto">guint <b>textureId</b> =
gst_gl_memory_get_texture_id(glmem);<br>
</div>
<div dir="auto">videoBuffer = new QGstVideoBuffer(buffer,
m_videoInfo, m_format.handleType(), <b>textureId</b>); <b
style="font-family:sans-serif"><br>
</b></div>
<div dir="auto"><br>
</div>
<div dir="auto">So I thought that things were starting to get
clear. Thus I decided to use vcsm to map the texture and
speed things up .</div>
<div dir="auto"><br>
</div>
<div dir="auto">First, on my <b>QAbstractVideoSurface::start</b>
implemented method I initialized vcsm and created a
EGLimageKHR.</div>
<div dir="auto"><br>
</div>
<div dir="auto">int w = Util::nextPOT(size.width());</div>
<div dir="auto">int h = Util::nextPOT(size.height());</div>
<div dir="auto">const EGLint attrib[] = {
EGL_IMAGE_PRESERVED_KHR, EGL_TRUE, EGL_NONE, EGL_NONE
};<br>
</div>
<div dir="auto">vcsm_info.width = w;<br>
</div>
<div dir="auto">
<div dir="auto">vcsm_info.height = h;</div>
<br>
</div>
<div dir="auto">vcsm_init();</div>
<div dir="auto">eglFbImage =
eglCreateImageKHR(eglGetCurrentDisplay(), EGL_NO_CONTEXT,
EGL_IMAGE_BRCM_VCSM, &vcsm_info, attrib); </div>
<div dir="auto"><br>
</div>
<div dir="auto">Second, on my <b>QAbstractVideoSurface::present</b>
implementation method I tried to grab the passed Texture
content, but it didn't work. Despite this, all OpenGL calls
succeeded. I can access the mapped buffer, but it doesn't
contain anything meaningful.<br>
</div>
<div dir="auto"><br>
</div>
<div dir="auto">QOpenGLFunctions* f = ctx->functions(); </div>
<div dir="auto">GLuint framebuffer; </div>
<div dir="auto">GLuint depthRenderbuffer; </div>
<div dir="auto">GLint prevFbo;</div>
<div dir="auto">GLenum status = GL_FRAMEBUFFER_COMPLETE;</div>
<div dir="auto">GLuint texture = static_cast<GLuint>(
currentFrame.handle().toInt() );</div>
<div dir="auto">int texWidth =
Util::nextPOT(currentFrame.width());</div>
<div dir="auto">int texHeight =
Util::nextPOT(currentFrame.height());</div>
<div dir="auto"> </div>
<div dir="auto">GLCHK(f->glGetIntegerv(
GL_FRAMEBUFFER_BINDING, &prevFbo )); </div>
<div dir="auto">GLCHK(f->glGenFramebuffers(1,
&framebuffer));</div>
<div dir="auto">GLCHK(f->glBindFramebuffer(GL_FRAMEBUFFER,
framebuffer));</div>
<div dir="auto">GLCHK(glActiveTexture(GL_TEXTURE0));</div>
<div dir="auto">GLCHK(glBindTexture(GL_TEXTURE_2D, texture));</div>
<div dir="auto">GLCHK(glTexParameteri(GL_TEXTURE_2D,
GL_TEXTURE_MAG_FILTER, GL_NEAREST));</div>
<div dir="auto">GLCHK(glTexParameteri(GL_TEXTURE_2D,
GL_TEXTURE_MIN_FILTER, GL_NEAREST)); </div>
<div dir="auto">GLCHK(glEGLImageTargetTexture2DOES(GL_TEXTURE_2D,
eglFbImage)); </div>
<div dir="auto">GLCHK(f->glFramebufferTexture2D(GL_FRAMEBUFFER,
GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, texture, 0)); </div>
<div dir="auto">GLCHK(glBindTexture(GL_TEXTURE_2D, 0)); </div>
<div dir="auto">status =
glCheckFramebufferStatus(GL_FRAMEBUFFER);</div>
<div dir="auto">// check fbo status </div>
<div dir="auto">GLCHK(f->glFinish()); </div>
<div dir="auto"><br>
</div>
<div dir="auto">uint8_t *vcsmBuffer; </div>
<div dir="auto">VCSM_CACHE_TYPE_T cacheType; </div>
<div dir="auto">vcsmBuffer =
(uint8_t*)vcsm_lock_cache(vcsm_info.vcsm_handle,
VCSM_CACHE_TYPE_HOST, &cacheType);</div>
<div dir="auto">// print buffer</div>
<div dir="auto">vcsm_unlock_ptr(vcsmBuffer); </div>
<div dir="auto"><br>
</div>
<div dir="auto">Is the Texture Id correctly filled by
omxh264dec or glupload ?</div>
<div dir="auto">Can i bind the texture id to a different fbo ?</div>
<div dir="auto">Why <b style="font-family:sans-serif">gst_video_frame_map
</b>correctly map the video frame content and vcsm do not
despite being addressable?</div>
<div dir="auto"><b style="font-family:sans-serif"><br>
</b></div>
<div dir="auto">If someone could give a tip on this I would be
thankful.<br style="border:none;margin:0px;padding:0px">
</div>
<div dir="auto"><br>
</div>
<div>regards.</div>
<div dir="auto"><br>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
gstreamer-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:gstreamer-devel@lists.freedesktop.org">gstreamer-devel@lists.freedesktop.org</a>
<a class="moz-txt-link-freetext" href="https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel">https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel</a>
</pre>
</blockquote>
<br>
</body>
</html>