Introduction and updates from NVIDIA

Tue Mar 22 21:43:18 UTC 2016

On Tue, Mar 22, 2016 at 01:49:59PM +0000, Daniel Stone wrote:
> On 21 March 2016 at 16:28, Miguel Angel Vico <mvicomoya at nvidia.com> wrote:
> I'd like to look at the elephant in the room, which is why you're
> using this in the first place (aside from general NVIDIA enthusiasm
> for encapsulating everything within EGL Streams/Output/Device/etc,
> dating back many years). Andy/Aaron, you've said that you found GBM to
> be inadequate, and I'd like to find out explicitly how. Through a few
> snippets of IRC and NVIDIA devtalk, so far I can see:
> 
> 'We can't choose an optimal rendering configuration, because we don't
> know how it's going to be used' - (almost completely) untrue. The FD
> you pass to gbm_device_create is that of the KMS device, a gbm_surface
> contains information as to how the plane (primary or overlay) will be
> configured, and an EGLDisplay lets you tie the rendering and scanout
> devices together. What more information do you need? It's true that we
> don't have a way to select individual rendering devices at the moment,
> but as said earlier, passing an EGLDevice as an attrib to
> GetPlatformDisplay would resolve that, as you would have the render
> device identified by the EGLDevice and the scanout device identified
> by the gbm_device. At that point, you have the full pipeline and can
> determine the optimal configuration.
> 
> 'We don't know when to schedule decompression, because there's no
> explicit barrier' - completely untrue. eglSwapBuffers is that barrier.
> For example, in Freescale i.MX6, the Vivante GPU and Freescale IPU
> (display controller) do not share a single common format between GPU
> render targets and IPU scanout sources, so require a mandatory
> detiling pass in between render and display. These work just fine with
> gbm with that pass scheduled by eglSwapBuffers. This to me seems
> completely explicit, unless there was something else you were meaning
> ... ?

There's display engines which can directly scan out buffers compressed by
the render engine. It's awesome, except randomly limited, so you need a
communication backchannel from your display driver all the way to your
buffer allocator thing on the client side. And depending upon luck you
really can't tell who should do the decompress past for most optimal
result upfront.

I think on android the most common way to do that is to attach arbitrary
metadata with a hand-rolled ioctl to dma-buf fds, which is ofc horrible.
Imo the right way is to create a real platform and start to standardize
some of this stuff (fb modifier) more, so that we can pass it from kms to
gbm, then to compositor clients through either a generic transport of
private extensions. Or maybe we can mostly hide all that.

> 'Width, height, pitch and format aren't enough information' - this is
> true, but not necessarily relevant. I'm not sure what the source of
> this actually is: is it the gbm_bo_get_*() APIs? If so, yes, they need
> to be extended with a gbm_bo_get_modifier() call, which would allow
> you to get the DRM format modifier to describe tiling/compression/et
> al (as well as perhaps being extended to allow you to extract multiple
> buffers/planes, e.g. to attach auxiliary compression buffers). If it's
> not gbm, what actually is it? The only other place I can think of
> (suggested by Pekka, I think) was the wl_drm protocol, which it should
> be stressed is a) not required in any way by Wayland, b) not a
> published/public protocol, c) not a stable protocol. wl_drm just
> happens to be the way that Mesa shares buffers, just as wl_viv is how
> Vivante's proprietary driver shares buffers, and mali_buffer_sharing
> is how the Mali driver does it. Since the server side is bound by
> eglBindWaylandDisplayWL and the client side is also only used through
> EGL, there is _no_ requirement for you to also implement wl_drm. As it
> is a hidden private Mesa protocol, there is also no requirement for
> the protocol to remain stable.

So I've what our own android folks all transport, and I think most of it
we can transport with the current addfb2.1 kms metadata. And we could even
add hints that kms atomic returns if a plane doesn't work with the most
preferred format that would just work in this config. Thus far I've
stumbled over 2 cases:
- compression formats that can't be easily described in addfb2.1 because
  they allocate a side buffer in some fancy special memory. The solution
  for that that was discussed at xdc2014 was to use a dma-buf to wrap that
  up, and then use as aux buffer (there's patches floating for that) with
  normal addfb2.1.
- content protection. Can't talk about this, but worst case it can all be
  captured in special-purpose buffers too I think.

> 'EGLStreams is the direction taken in Vulkan' - I would argue not. IMO
> the explicit buffer management on the client side does not parallel
> EGLStreams, and notably there is no equivalent consumer interface
> offered on the server side, but instead the individual-buffer-driven
> approach is taken. It's true that VK_WSI_display_swapchain does exist
> and does match the EGLStreams model fairly closely, but also that it
> does not have universal implementation: the Intel 'anv' Mesa-based
> driver does not implement display_swapchain, instead having an
> interface to export a VkImage as a dmabuf. It's true that the latter
> is not optimal (it lacks the explicit targeting required to determine
> the most optimal tiling/compression strategy), but OTOH it is
> precedent for explicitly avoiding the
> VK_WSI_display_swapchain/EGLStreams model for Vulkan on KMS, just as
> GBM avoids it for EGL on KMS.

I'm not sure a swapchain/stream is good enough, since the trouble really
starts when you have tons of hw planes and changing configurations.
Looking at individual streams instead of the global state is pointless in
that case.

Same for atomic, syncing multiple streams looks pretty tricky. And iirc
when I pinged Jakob Bornecrantz (who seems to know/like streams somewhat)
there's no way to eachive that.

> I think it's been good to have this series to push the discussion
> further in more concrete terms, but unfortunately I have to say that
> I'm even less convinced now than I have ever been. Sorry.

Well the thing that irks me is that this isn't aiming to build a common
platform. There's definitely issues with gbm/gralloc+kms+egl in upstream
repos, and vendors have hacked around those in all kinds of horrible ways.
But trying to fix this mess with yet another vendor-private solution just
doesn't help. Instead we need to fix what is there, for everyone, instead
of fragmenting more.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch