Readback OpenGL texture from omxh264dec

[Matthew Waters]

The OpenGL texture out of omxh264dec on the RPi is wrapped in a
GstGLMemoryEGL which already wraps the EGLImage produced by the OMX
decoder.  I'm not entirely sure what VCSM is or how it actually relates
to all this but I assume that the following pipeline works for you?
'omxh264dec ! glimagesinkelement'  If so, that is already using the
EGLImage/OpenGL texture produced by omxh264dec and rendering using
OpenGL with glimagesink.  It is very hard in general for the RPi to do
anything but only decode and display a 1080p at 30 video and even then it
can just barely do it.

On 21/6/20 10:22 pm, Rafael Savignon wrote:
> Hi all,
> I'm developing a Qt5 application to playback hardware decoded h.264
> files. But on raspberry pi 3 its struggling to play media greater than
> 720p. After some investigation of *QGstVideoBuffer* class I realized
> that calls to gst_video_frame_map function passing bigger frames were
> consuming much cpu time ~50ms, which was resulting in the postponing
> of next frames delivery. So I had to find out a more performatic way
> to read the video frame content. After some search I found out that a
> special buffer mechanism could enable applications to map GPU memory
> directly to addressable process memory, avoiding unnecessary copy
> (VCSM). Then I took another look at QtMultimedia source code and saw
> that a OpenGL texture handle was being packaged in 
> with *QGstVideoBuffer*, as shown below.
> guint *textureId* = gst_gl_memory_get_texture_id(glmem);
> videoBuffer = new QGstVideoBuffer(buffer, m_videoInfo,
> m_format.handleType(), *textureId*); *
> *
>
> So I thought that things were starting to get clear. Thus I decided to
> use vcsm to map the texture and speed things up .
>
> First, on my *QAbstractVideoSurface::start* implemented method I
> initialized vcsm and created a EGLimageKHR.
>
> int w = Util::nextPOT(size.width());
> int h = Util::nextPOT(size.height());
> const EGLint attrib[] = { EGL_IMAGE_PRESERVED_KHR, EGL_TRUE, EGL_NONE,
> EGL_NONE };
> vcsm_info.width = w;
> vcsm_info.height = h;
>
> vcsm_init();
> eglFbImage = eglCreateImageKHR(eglGetCurrentDisplay(), EGL_NO_CONTEXT,
> EGL_IMAGE_BRCM_VCSM, &vcsm_info, attrib); 
>
> Second, on my *QAbstractVideoSurface::present*  implementation method
> I tried to grab the passed Texture content, but it didn't work.
> Despite this, all OpenGL calls succeeded. I can access the mapped
> buffer, but it doesn't contain anything meaningful.
>
> QOpenGLFunctions* f = ctx->functions(); 
> GLuint framebuffer; 
> GLuint depthRenderbuffer; 
> GLint prevFbo;
> GLenum status = GL_FRAMEBUFFER_COMPLETE;
> GLuint texture = static_cast<GLuint>( currentFrame.handle().toInt() );
> int texWidth = Util::nextPOT(currentFrame.width());
> int texHeight = Util::nextPOT(currentFrame.height());
>   
> GLCHK(f->glGetIntegerv( GL_FRAMEBUFFER_BINDING, &prevFbo )); 
> GLCHK(f->glGenFramebuffers(1, &framebuffer));
> GLCHK(f->glBindFramebuffer(GL_FRAMEBUFFER, framebuffer));
> GLCHK(glActiveTexture(GL_TEXTURE0));
> GLCHK(glBindTexture(GL_TEXTURE_2D, texture));
> GLCHK(glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST));
> GLCHK(glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST)); 
> GLCHK(glEGLImageTargetTexture2DOES(GL_TEXTURE_2D, eglFbImage)); 
> GLCHK(f->glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
> GL_TEXTURE_2D, texture, 0)); 
> GLCHK(glBindTexture(GL_TEXTURE_2D, 0)); 
> status = glCheckFramebufferStatus(GL_FRAMEBUFFER);
> // check fbo status 
> GLCHK(f->glFinish()); 
>
> uint8_t *vcsmBuffer; 
> VCSM_CACHE_TYPE_T cacheType; 
> vcsmBuffer = (uint8_t*)vcsm_lock_cache(vcsm_info.vcsm_handle,
> VCSM_CACHE_TYPE_HOST, &cacheType);
> // print buffer
> vcsm_unlock_ptr(vcsmBuffer); 
>
> Is the Texture Id correctly filled by omxh264dec or glupload ?
> Can i bind the texture id to a different fbo ?
> Why *gst_video_frame_map *correctly map the video frame content and
> vcsm do not despite being addressable?
> *
> *
> If someone could give a tip on this I would be thankful.
>
> regards.
>
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20200622/c29677fd/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20200622/c29677fd/attachment-0001.sig>