Gstreamer + Clutter + Wayland

Sat Apr 6 19:54:23 UTC 2019

Le sam. 6 avr. 2019 06 h 10, horai <ivo.hora at seznam.cz> a écrit :

> Nicolas Dufresne-5 wrote
> > Le jeudi 28 mars 2019 à 10:57 -0500, horai a écrit :
> >
> >
> > omxh264dec zero-copy solution is not supported by the Mesa VC4 driver,
> > only the brcm GLES driver supports this. I know they are working on a
> > V4L2 driver that speaks mmal in the kernel, with DMABuf plumbed through
> > mmal in order to allow using standard DMABuf import path into GL. That
> > being said, patches where submitted to cluttersink (which is not part
> > of GStreamer project), but was never merged due to lack of proper
> > support in CoGL/Clutter. Clutter is basically left without maintenance
> > these days.
> >
> > The right solution is what is coming next, but I have no idea when this
> > will happen, or if there is a way to pick dev version here and there to
> > make it work.
>
> Dear sir,
>
> Would be so kind and help me understand that matter clearly?
> I had the impression that omxh264dec zero-copy solution is related to
> decoding part only and I thought that this process is not dependent on the
> display backend. From your words I understand that the zero-copy slow down
> problem of MESA VC4 is present in X11 as well as in Wayland, moreover I
> think that Wayland does not work without VC4 at all.So, I understand that
> if
> my application would be using a sink utilizing VC4 (in my case clutter
> stage
> in GTK panel), I would definitely face the zero-copy problem. But this is
> not dependent on X11 or Wayland.
> Based upon my experience,when I started using VC4 openGL driver in X11, I
> experienced huge performance boost in my application (gstreamer + GTK +
> clutter), therefore I blamed clutter as the bottleneck in my application
> and
> I somehow presumed that clutter switched from utilizing OpenGL ES 2.0 to
> OpenGL driver internally and increased performance.
> I do not fully understand the structure of VC4, I understand VC4 as a
> driver
> of Broadcom videocore GPU driver exposing the hardware to software via
> OpenGL ES 1.0/2.0 API as well as OpenGL 2.1 API (on a hardware level it is
> OpenGL ES 2.0 hardware renderer and the OpenGL is just a software emulation
> layer for OpenGL 2.1 specification functions which are missing in OpenGL ES
> 2.0, this could be achieved only because of the fact that OpenGL ES 2.0 was
> derived from OpenGL 2.1 therefore they are very much close to each other)
> and due to the fact that Clutter supports OpenGL as well as OpenGL
> ES(either
> 1.0/2.0) Clutter can choose between OpenGL and OpenGL ES, because the
> libclutter library could be compiled with OpenGL (and I assume that
> Raspbian
> has binary library compiled with both flags enabled during configuration
> process) as well as OpenGL ES support. Therefore I presumed that when
> starting using VC4, Clutter layer in my application stopped using OpenGL ES
> and switched to OpenGL and that increased the performance or maybe my
> understanding of the matter could be completely wrong and VC4 could be
> simply much more optimized than original Broadcom OpenGL ES 2.0 binary
> driver and therefore Clutter could be internally using OpenGL ES 2.0 (no
> OpenGL) but VC4 is much faster then original Broadcom Open GL ES 2.0 binary
> driver but lacks zero-copy solution which has no relation to Clutter it
> relates to omx codec encoding/decoding video speaking in gtreamer words for
> omxh264dec element.
> In short despite the fact I am not using zero-copy solution because of VC4
> usage I gained performance boost in video rendering (displaying) via
> Clutter
> layer when using VC4.
> My problem was, why do I then experince such a slow rendering when using
> Wayland with Clutter?
> Please, would you be so kind and bring some more light to my understanding
> of the matter as I presumed that omxh264dec has no relation to VC4 OpenGL
> ES
> 2.0, I assumed that is does only video decoding (for example h264) on GPU,
> but I understand that for decoding it also needs to use some part of VC4
> driver because GPU driver is responsible for video processing as well as
> for
> video or image displaying but I assumed that encoding and displaying use
> totally different VC4 APIs.
> I do not know what is being used from VC4 for video encoding/decoding but I
> assumed that for displaying (in my case clutter stage embedded into GTK
> panel on top of Wayland or X11 utilizing VC4) OpenGL2.1 or OpenGLES 2.0 API
> is used specifically talking about RPI 3B+ and Broadcom BCM2837 GPU)
>

I've just stumbled across this thread, and it seems that v4l2 mem2mem codec
has landed already, and GStreamer 1.14 has been ship for this purpose.
Basically means you should stop using OMX and try v4l2h264dec element
instead.

https://www.raspberrypi.org/forums/viewtopic.php?t=234778

Cluttersink does not support dmabuf, so it will still require a bit of
copies, but with that driver you save a dma copy from vcore. I hope it will
compensate. You could also try your luck with adding glupload in front, but
I'm not sure what is the state of the legacy gluploadmeta that this would
use.

When you moved to X11/Clutter what you did was to turn software color
conversion into GPU accelerated conversion and that certainly give a boost.
When you then move to using a compositor (over X11 or Wayland) you start
rendering each window twice, once offscreen and finally composed to screen.
This requires more GPU but I would also suspect more memory (see kernel
parameter gpu_mem). Try increasing that amount of memory, something like
128M, and see if that helps.

>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190406/8bf16db6/attachment-0001.html>