Readback OpenGL texture from omxh264dec

[Rafael Savignon]

Hi all,
I'm developing a Qt5 application to playback hardware decoded h.264 files.
But on raspberry pi 3 its struggling to play media greater than 720p. After
some investigation of *QGstVideoBuffer* class I realized that calls
to gst_video_frame_map function passing bigger frames were consuming much
cpu time ~50ms, which was resulting in the postponing of next frames
delivery. So I had to find out a more performatic way to read the video
frame content. After some search I found out that a special buffer
mechanism could enable applications to map GPU memory directly to
addressable process memory, avoiding unnecessary copy (VCSM). Then I took
another look at QtMultimedia source code and saw that a OpenGL texture
handle was being packaged in  with *QGstVideoBuffer*, as shown below.

guint *textureId* = gst_gl_memory_get_texture_id(glmem);
videoBuffer = new QGstVideoBuffer(buffer, m_videoInfo,
m_format.handleType(), *textureId*);

So I thought that things were starting to get clear. Thus I decided to use
vcsm to map the texture and speed things up .

First, on my *QAbstractVideoSurface::start* implemented method I
initialized vcsm and created a EGLimageKHR.

int w = Util::nextPOT(size.width());
int h = Util::nextPOT(size.height());
const EGLint attrib[] = { EGL_IMAGE_PRESERVED_KHR, EGL_TRUE, EGL_NONE,
EGL_NONE };
vcsm_info.width = w;
vcsm_info.height = h;

vcsm_init();
eglFbImage = eglCreateImageKHR(eglGetCurrentDisplay(), EGL_NO_CONTEXT,
EGL_IMAGE_BRCM_VCSM, &vcsm_info, attrib);

Second, on my *QAbstractVideoSurface::present*  implementation method I
tried to grab the passed Texture content, but it didn't work. Despite this,
all OpenGL calls succeeded. I can access the mapped buffer, but it doesn't
contain anything meaningful.

QOpenGLFunctions* f = ctx->functions();
GLuint framebuffer;
GLuint depthRenderbuffer;
GLint prevFbo;
GLenum status = GL_FRAMEBUFFER_COMPLETE;
GLuint texture = static_cast<GLuint>( currentFrame.handle().toInt() );
int texWidth = Util::nextPOT(currentFrame.width());
int texHeight = Util::nextPOT(currentFrame.height());

GLCHK(f->glGetIntegerv( GL_FRAMEBUFFER_BINDING, &prevFbo ));
GLCHK(f->glGenFramebuffers(1, &framebuffer));
GLCHK(f->glBindFramebuffer(GL_FRAMEBUFFER, framebuffer));
GLCHK(glActiveTexture(GL_TEXTURE0));
GLCHK(glBindTexture(GL_TEXTURE_2D, texture));
GLCHK(glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST));
GLCHK(glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST));
GLCHK(glEGLImageTargetTexture2DOES(GL_TEXTURE_2D, eglFbImage));
GLCHK(f->glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
GL_TEXTURE_2D, texture, 0));
GLCHK(glBindTexture(GL_TEXTURE_2D, 0));
status = glCheckFramebufferStatus(GL_FRAMEBUFFER);
// check fbo status
GLCHK(f->glFinish());

uint8_t *vcsmBuffer;
VCSM_CACHE_TYPE_T cacheType;
vcsmBuffer = (uint8_t*)vcsm_lock_cache(vcsm_info.vcsm_handle,
VCSM_CACHE_TYPE_HOST, &cacheType);
// print buffer
vcsm_unlock_ptr(vcsmBuffer);

Is the Texture Id correctly filled by omxh264dec or glupload ?
Can i bind the texture id to a different fbo ?
Why *gst_video_frame_map *correctly map the video frame content and vcsm do
not despite being addressable?

If someone could give a tip on this I would be thankful.

regards.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20200621/28c8afed/attachment.htm>