Gstreamer + Clutter + Wayland

[horai]

Nicolas Dufresne-5 wrote
> Le jeudi 28 mars 2019 à 10:57 -0500, horai a écrit :
> 
> 
> omxh264dec zero-copy solution is not supported by the Mesa VC4 driver,
> only the brcm GLES driver supports this. I know they are working on a
> V4L2 driver that speaks mmal in the kernel, with DMABuf plumbed through
> mmal in order to allow using standard DMABuf import path into GL. That
> being said, patches where submitted to cluttersink (which is not part
> of GStreamer project), but was never merged due to lack of proper
> support in CoGL/Clutter. Clutter is basically left without maintenance
> these days.
> 
> The right solution is what is coming next, but I have no idea when this
> will happen, or if there is a way to pick dev version here and there to
> make it work.

Dear sir,

Would be so kind and help me understand that matter clearly?
I had the impression that omxh264dec zero-copy solution is related to
decoding part only and I thought that this process is not dependent on the
display backend. From your words I understand that the zero-copy slow down
problem of MESA VC4 is present in X11 as well as in Wayland, moreover I
think that Wayland does not work without VC4 at all.So, I understand that if
my application would be using a sink utilizing VC4 (in my case clutter stage
in GTK panel), I would definitely face the zero-copy problem. But this is
not dependent on X11 or Wayland.
Based upon my experience,when I started using VC4 openGL driver in X11, I
experienced huge performance boost in my application (gstreamer + GTK +
clutter), therefore I blamed clutter as the bottleneck in my application and
I somehow presumed that clutter switched from utilizing OpenGL ES 2.0 to
OpenGL driver internally and increased performance.
I do not fully understand the structure of VC4, I understand VC4 as a driver
of Broadcom videocore GPU driver exposing the hardware to software via
OpenGL ES 1.0/2.0 API as well as OpenGL 2.1 API (on a hardware level it is
OpenGL ES 2.0 hardware renderer and the OpenGL is just a software emulation
layer for OpenGL 2.1 specification functions which are missing in OpenGL ES
2.0, this could be achieved only because of the fact that OpenGL ES 2.0 was
derived from OpenGL 2.1 therefore they are very much close to each other)
and due to the fact that Clutter supports OpenGL as well as OpenGL ES(either
1.0/2.0) Clutter can choose between OpenGL and OpenGL ES, because the
libclutter library could be compiled with OpenGL (and I assume that Raspbian
has binary library compiled with both flags enabled during configuration
process) as well as OpenGL ES support. Therefore I presumed that when
starting using VC4, Clutter layer in my application stopped using OpenGL ES
and switched to OpenGL and that increased the performance or maybe my
understanding of the matter could be completely wrong and VC4 could be
simply much more optimized than original Broadcom OpenGL ES 2.0 binary
driver and therefore Clutter could be internally using OpenGL ES 2.0 (no
OpenGL) but VC4 is much faster then original Broadcom Open GL ES 2.0 binary
driver but lacks zero-copy solution which has no relation to Clutter it
relates to omx codec encoding/decoding video speaking in gtreamer words for
omxh264dec element.
In short despite the fact I am not using zero-copy solution because of VC4
usage I gained performance boost in video rendering (displaying) via Clutter
layer when using VC4. 
My problem was, why do I then experince such a slow rendering when using
Wayland with Clutter?
Please, would you be so kind and bring some more light to my understanding
of the matter as I presumed that omxh264dec has no relation to VC4 OpenGL ES
2.0, I assumed that is does only video decoding (for example h264) on GPU,
but I understand that for decoding it also needs to use some part of VC4
driver because GPU driver is responsible for video processing as well as for
video or image displaying but I assumed that encoding and displaying use
totally different VC4 APIs.
I do not know what is being used from VC4 for video encoding/decoding but I
assumed that for displaying (in my case clutter stage embedded into GTK
panel on top of Wayland or X11 utilizing VC4) OpenGL2.1 or OpenGLES 2.0 API
is used specifically talking about RPI 3B+ and Broadcom BCM2837 GPU)



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/