[Bug 711155] wayland: add wl_drm support to wayland sink

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=711155

Nicolas Dufresne (stormer) <nicolas.dufresne at collabora.co.uk> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |nicolas.dufresne at collabora.
                   |                            |co.uk

--- Comment #15 from Nicolas Dufresne (stormer) <nicolas.dufresne at collabora.co.uk> ---
The dmabuf protocol is now merged in wayland, which re-open this subject. Note
that dmabuf currently has few limitation we need to take into account. The main
one in my opinion:

- No listing of supported color formats (because it's hidden inside the EGL
stack)
- Required video alignment and allocation type (CMA, SG, VM) is unknown

That simply means there is no guaranty that decoder driver produced buffer will
work unless you have an "harmonized" allocation for your board integration.
This is just a point in time fact, this could eventually change.

In the end, wayland currently offer 3 mainline protocol for creating wl_buffer.
In the shm case, we have an dedicated allocation which works mostly like any
normal allocator. We know the list of formats and we have the option to copy
non-shm memory into shm. Most compositor only offset RGB based formats. Any
other format would require video conversion before this sink. This is similar
to ximagesink in some ways.

The second is the EGL protocol. This one is defined by Mesa, and requires the
wayland client to do the upload (so the compositor don't get overloaded). This
can be achieve using libgstgl, but the recommended way is to wrap such sink
into glsinkbin. This is vaguely the same thing as glimagesink over wayland. I
don't know myself what could be optimized by the compositor itself.

The third is dmabuf protocol. It's a black box, but it's know that an
unsupported dmabuf will fail in the creation of the wl_buffer if anything is
not compatible. This element require no transformation but present a
uncertainty on let's say generic systems (not to say desktop system). While it
will just work on a fixed system where all driver have been tuned to work
together or where ION has been tuned and is used for allocation.

Assuming we target all these possibilities, it becomes clear that the
negotiation for each case is largely different. Building a single element for
all those use cases in one will create complexity and might require pulling
libvideoconvert and libgstgl. It also removes some flexibility as I believe
there is use cases where you want DMABUF or nothing and want to tune the system
to make it work this way.

For this reason, I'd like to propose splitting up this in a set of seperate
elements (with a base class of course). We'd have eventually 3 sinks:

  waylandshmsink
  waylandeglsink
  waylanddmabufsink

In addition to that, to cover the generic case, where we want to try everything
to make it work regardless the price (like on desktop), we would have
waylandsinkbin that will know the specificity of each internal sink. And will
allow doing fallback when dmabuf and egl failed. The exposed list of color
format would be the list of format supported by videoconvert. Which I believe
is a fair limitation on generic/desktop platforms.

This design could decrease the complexity of each sink, making it very
efficient to use in embedded/controlled use cases, which to be fair will likely
be the biggest users of these element. On desktop, I do believe most use case
will be satisfied by gl base sink, specially the ones implementing easy to use
widgets like gtkglsink and qmlglsink.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.