[Mesa-dev] Allocator Nouveau driver, Mesa EXT_external_objects, and DRM metadata import interfaces

[Miguel Angel Vico]

Hi,

Besides the DRM modifiers discussion in the other forks or this thread
(I should've probably started separate threads), has anyone gotten the
chance to look at least at the mesa changes and allocator changes I
shared below?

With respect to Mesa changes, I think it might be worth merging the
EXT_external_objects nouveau implementation upstream. Should I just
send the list of patches as a formal RFR?

With respect to Allocator changes, it'd be nice getting someone else's
(out of NVIDIA) feedback.

Thanks.

On Wed, 20 Dec 2017 08:51:51 -0800
Miguel Angel Vico <mvicomoya at nvidia.com> wrote:

> Hi all,
> 
> As many of you already know, I've been working with James Jones on the
> Generic Device Allocator project lately. He started a discussion thread
> some weeks ago seeking feedback on the current prototype of the library
> and advice on how to move all this forward, from a prototype stage to
> production. For further reference, see:
> 
>    https://lists.freedesktop.org/archives/mesa-dev/2017-November/177632.html
> 
> From the thread above, we came up with very interesting high level
> design ideas for one of the currently missing parts in the library:
> Usage transitions. That's something I'll personally work on during the
> following weeks.
> 
> 
> In the meantime, I've been working on putting together an open source
> implementation of the allocator mechanisms using the Nouveau driver for
> all to be able to play with.
> 
> Below I'm seeking feedback on a bunch of changes I had to make to
> different components of the graphics stack:
> 
> ** Allocator **
> 
>   An allocator driver implementation on top of Nouveau. The current
>   implementation only handles pitch linear layouts, but that's enough
>   to have the kmscube port working using the allocator and Nouveau
>   drivers.
> 
>   You can pull these changes from
> 
>       https://github.com/mvicomoya/allocator/tree/wip/mvicomoya/nouveau-driver
> 
> ** Mesa **
> 
>   James's kmscube port to use the allocator relies on the
>   EXT_external_objects extension to import allocator allocations to
>   OpenGL as a texture object. However, the Nouveau implementation of
>   these mechanisms is missing in Mesa, so I went ahead and added them.
> 
>   You can pull these changes from
> 
>       https://github.com/mvicomoya/mesa/tree/wip/mvicomoya/EXT_external_objects-nouveau
> 
>   Also, James's kmscube port uses the NVX_unix_allocator_import
>   extension to attach allocator metadata to texture objects so the
>   driver knows how to deal with the imported memory.
> 
>   Note that there isn't a formal spec for this extension yet. For now,
>   it just serves as an experimental mechanism to import allocator
>   memory in OpenGL, and attach metadata to texture objects.
> 
>   You can pull these changes (written on top of the above) from:
> 
>       https://github.com/mvicomoya/mesa/tree/wip/mvicomoya/NVX_unix_allocator_import
> 
> ** kmscube **
> 
>   Mostly minor fixes and improvements on top of James's port to use the
>   allocator. Main thing is the allocator initialization path will use
>   EGL_MESA_platform_surfaceless if EGLDevice platform isn't supported
>   by the underlying EGL implementation.
> 
>   You can pull these changes from:
> 
>       https://github.com/mvicomoya/kmscube/tree/wip/mvicomoya/allocator-nouveau
> 
> 
> With all the above you should be able to get kmscube working using the
> allocator on top of the Nouveau driver.
> 
> 
> Another of the missing pieces before we can move this to production is
> importing allocations to DRM FB objects. This is probably one of the
> most sensitive parts of the project as it requires modification/addition
> of kernel driver interfaces.
> 
> At XDC2017, James had several hallway conversations with several people
> about this, all having different opinions. I'd like to take this
> opportunity to also start a discussion about what's the best option to
> create a path to get allocator allocations added as DRM FB objects.
> 
> These are the few options we've considered to start with:
> 
>   A) Have vendor-private ioctls to set properties on GEM objects that
>      are inherited by the FB objects. This is how our (NVIDIA) desktop
>      DRM driver currently works. This would require every vendor to add
>      their own ioctl to process allocator metadata, but the metadata is
>      actually a vendor-agnostic object more like DRM modifiers. We'd
>      like to come up with a vendor-agnostic solutions that can be
>      integrated to core DRM.
> 
>   B) Add a new drmModeAddFBWithMetadata() command that takes allocator
>      metadata blobs for each plane of the FB. Some people in the
>      community have mentioned this is their preferred design. This,
>      however, means we'd have to go through the exercise of adding
>      another metadata mechanism to the whole graphics stack.
> 
>   C) Shove allocator metadata into DRM by defining it to be a separate
>      plane in the image, and using the existing DRM modifiers mechanism
>      to indicate there is another plane for each "real" plane added. It
>      isn't clear how this scales to surfaces that already need several
>      planes, but there are some people that see this as the only way
>      forward. Also, we would have to create a separate GEM buffer for
>      the metadatada itself, which seems excessive.
> 
> We personally like option (B) better, and have already started to
> prototype the new path (which is actually very similar to the
> drmModeAddFB2() one). You can take a look at the new interfaces here:
> 
>     https://github.com/mvicomoya/linux/tree/wip/mvicomoya/drm_addfb_with_metadata__4.14-rc8
> 
> There may be other options that haven't been explored yet that could be
> a better choice than the above, so any suggestion will be greatly
> appreciated.
> 
> 
> Thanks,
> 


-- 
Miguel