[Mesa-dev] Allocator Nouveau driver, Mesa EXT_external_objects, and DRM metadata import interfaces

Wed Dec 20 19:54:10 UTC 2017

On Wed, Dec 20, 2017 at 11:51 AM, Daniel Vetter <daniel at ffwll.ch> wrote:
> Since this also involves the kernel let's add dri-devel ...
>
> On Wed, Dec 20, 2017 at 5:51 PM, Miguel Angel Vico <mvicomoya at nvidia.com> wrote:
>> Hi all,
>>
>> As many of you already know, I've been working with James Jones on the
>> Generic Device Allocator project lately. He started a discussion thread
>> some weeks ago seeking feedback on the current prototype of the library
>> and advice on how to move all this forward, from a prototype stage to
>> production. For further reference, see:
>>
>>    https://lists.freedesktop.org/archives/mesa-dev/2017-November/177632.html
>>
>> From the thread above, we came up with very interesting high level
>> design ideas for one of the currently missing parts in the library:
>> Usage transitions. That's something I'll personally work on during the
>> following weeks.
>>
>>
>> In the meantime, I've been working on putting together an open source
>> implementation of the allocator mechanisms using the Nouveau driver for
>> all to be able to play with.
>>
>> Below I'm seeking feedback on a bunch of changes I had to make to
>> different components of the graphics stack:
>>
>> ** Allocator **
>>
>>   An allocator driver implementation on top of Nouveau. The current
>>   implementation only handles pitch linear layouts, but that's enough
>>   to have the kmscube port working using the allocator and Nouveau
>>   drivers.
>>
>>   You can pull these changes from
>>
>>       https://github.com/mvicomoya/allocator/tree/wip/mvicomoya/nouveau-driver
>>
>> ** Mesa **
>>
>>   James's kmscube port to use the allocator relies on the
>>   EXT_external_objects extension to import allocator allocations to
>>   OpenGL as a texture object. However, the Nouveau implementation of
>>   these mechanisms is missing in Mesa, so I went ahead and added them.
>>
>>   You can pull these changes from
>>
>>       https://github.com/mvicomoya/mesa/tree/wip/mvicomoya/EXT_external_objects-nouveau
>>
>>   Also, James's kmscube port uses the NVX_unix_allocator_import
>>   extension to attach allocator metadata to texture objects so the
>>   driver knows how to deal with the imported memory.
>>
>>   Note that there isn't a formal spec for this extension yet. For now,
>>   it just serves as an experimental mechanism to import allocator
>>   memory in OpenGL, and attach metadata to texture objects.
>>
>>   You can pull these changes (written on top of the above) from:
>>
>>       https://github.com/mvicomoya/mesa/tree/wip/mvicomoya/NVX_unix_allocator_import
>>
>> ** kmscube **
>>
>>   Mostly minor fixes and improvements on top of James's port to use the
>>   allocator. Main thing is the allocator initialization path will use
>>   EGL_MESA_platform_surfaceless if EGLDevice platform isn't supported
>>   by the underlying EGL implementation.
>>
>>   You can pull these changes from:
>>
>>       https://github.com/mvicomoya/kmscube/tree/wip/mvicomoya/allocator-nouveau
>>
>>
>> With all the above you should be able to get kmscube working using the
>> allocator on top of the Nouveau driver.
>>
>>
>> Another of the missing pieces before we can move this to production is
>> importing allocations to DRM FB objects. This is probably one of the
>> most sensitive parts of the project as it requires modification/addition
>> of kernel driver interfaces.
>>
>> At XDC2017, James had several hallway conversations with several people
>> about this, all having different opinions. I'd like to take this
>> opportunity to also start a discussion about what's the best option to
>> create a path to get allocator allocations added as DRM FB objects.
>>
>> These are the few options we've considered to start with:
>>
>>   A) Have vendor-private ioctls to set properties on GEM objects that
>>      are inherited by the FB objects. This is how our (NVIDIA) desktop
>>      DRM driver currently works. This would require every vendor to add
>>      their own ioctl to process allocator metadata, but the metadata is
>>      actually a vendor-agnostic object more like DRM modifiers. We'd
>>      like to come up with a vendor-agnostic solutions that can be
>>      integrated to core DRM.
>>
>>   B) Add a new drmModeAddFBWithMetadata() command that takes allocator
>>      metadata blobs for each plane of the FB. Some people in the
>>      community have mentioned this is their preferred design. This,
>>      however, means we'd have to go through the exercise of adding
>>      another metadata mechanism to the whole graphics stack.
>>
>>   C) Shove allocator metadata into DRM by defining it to be a separate
>>      plane in the image, and using the existing DRM modifiers mechanism
>>      to indicate there is another plane for each "real" plane added. It
>>      isn't clear how this scales to surfaces that already need several
>>      planes, but there are some people that see this as the only way
>>      forward. Also, we would have to create a separate GEM buffer for
>>      the metadatada itself, which seems excessive.
>>
>> We personally like option (B) better, and have already started to
>> prototype the new path (which is actually very similar to the
>> drmModeAddFB2() one). You can take a look at the new interfaces here:
>>
>>     https://github.com/mvicomoya/linux/tree/wip/mvicomoya/drm_addfb_with_metadata__4.14-rc8
>>
>> There may be other options that haven't been explored yet that could be
>> a better choice than the above, so any suggestion will be greatly
>> appreciated.
>
> What kind of metadata are we talking about here? Addfb has tons of
> stuff already that's "metadata". The only thing I've spotted is
> PITCH_ALIGNMENT, which is maybe something we want drm drivers to tell
> userspace, but definitely not something addfb ever needs. addfb only
> needs the resulting pitch that we actually allocated (and might decide
> it doesn't like that, but that's a different issue).
>
> And since there's no patches for nouveau itself I can't really say
> anything beyond that.

I'd like to see concrete examples of actual display controllers
supporting more format layouts than what can be specified with a 64
bit modifier.

Kristian

> -Daniel
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> +41 (0) 79 365 57 48 - http://blog.ffwll.ch
> _______________________________________________
> dri-devel mailing list
> dri-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel