[Mesa-dev] GBM and the Device Memory Allocator Proposals

[Rob Clark]

On Mon, Nov 20, 2017 at 8:11 PM, James Jones <jajones at nvidia.com> wrote:
> As many here know at this point, I've been working on solving issues related
> to DMA-capable memory allocation for various devices for some time now.  I'd
> like to take this opportunity to apologize for the way I handled the EGL
> stream proposals.  I understand now that the development process followed
> there was unacceptable to the community and likely offended many great
> engineers.
>
> Moving forward, I attempted to reboot talks in a more constructive manner
> with the generic allocator library proposals & discussion forum at XDC 2016.
> Some great design ideas came out of that, and I've since been prototyping
> some code to prove them out before bringing them back as official proposals.
> Again, I understand some people are growing concerned that I've been doing
> this off on the side in a github project that has primarily NVIDIA
> contributors.  My goal was only to avoid wasting everyone's time with
> unproven ideas.  The intent was never to dump the prototype code as-is on
> the community and presume acceptance. It's just a public research project.
>
> Now the prototyping is nearing completion, and I'd like to renew discussion
> on whether and how the new mechanisms can be integrated with the Linux
> graphics stack.
>
> I'd be interested to know if more work is needed to demonstrate the
> usefulness of the new mechanisms, or whether people think they have value at
> this point.
>
> After talking with people on the hallway track at XDC this year, I've heard
> several proposals for incorporating the new mechanisms:
>
> -Include ideas from the generic allocator design into GBM.  This could take
> the form of designing a "GBM 2.0" API, or incrementally adding to the
> existing GBM API.
>
> -Develop a library to replace GBM.  The allocator prototype code could be
> massaged into something production worthy to jump start this process.
>
> -Develop a library that sits beside or on top of GBM, using GBM for
> low-level graphics buffer allocation, while supporting non-graphics kernel
> APIs directly.  The additional cross-device negotiation and sorting of
> capabilities would be handled in this slightly higher-level API before
> handing off to GBM and other APIs for actual allocation somehow.

tbh, I kinda see GBM and $new_thing sitting side by side.. GBM is
still the "winsys" for running on "bare metal" (ie. kms).  And we
don't want to saddle $new_thing with aspects of that, but rather have
it focus on being the thing that in multiple-"device"[1] scenarious
figures out what sort of buffer can be allocated by who for sharing.
Ie $new_thing should really not care about winsys level things like
cursors or surfaces.. only buffers.

The mesa implementation of $new_thing could sit on top of GBM,
although it could also just sit on top of the same internal APIs that
GBM sits on top of.  That is an implementation detail.  It could be
that GBM grows an API to return an instance of $new_thing for
use-cases that involve sharing a buffer with the GPU.  Or perhaps that
is exposed via some sort of EGL extension.  (We probably also need a
way to get an instance from libdrm (?) for display-only KMS drivers,
to cover cases like etnaviv sharing a buffer with a separate display
driver.)

[1] where "devices" could be multiple GPUs or multiple APIs for one or
more GPUs, but also includes non-GPU devices like camera, video
decoder, "image processor" (which may or may not be part of camera),
etc, etc


> -I have also heard some general comments that regardless of the relationship
> between GBM and the new allocator mechanisms, it might be time to move GBM
> out of Mesa so it can be developed as a stand-alone project.  I'd be
> interested what others think about that, as it would be something worth
> coordinating with any other new development based on or inside of GBM.

+1

We already have at least a couple different non-mesa implementations
of GBM (which afaict tend to lag behind mesa's GBM and cause
headaches).

The extracted part probably isn't much more than a header and shim.
But probably does need to grow some versioning for the backend to know
if, for example, gbm->bo_map() is supported.. at least it could
provide stubs that return an error, rather than having link-time fail
if building something w/ $vendor's old gbm implementation.

> And of course I'm open to any other ideas for integration.  Beyond just
> where this code would live, there is much to debate about the mechanisms
> themselves and all the implementation details.  I was just hoping to kick
> things off with something high level to start.

My $0.02, is that the place where devel happens and place to go for
releases could be different.  Either way, I would like to see git tree
for tagged release versions live on fd.o and use the common release
process[2] for generating/uploading release tarballs that distros can
use.

[2] https://cgit.freedesktop.org/xorg/util/modular/tree/release.sh

> For reference, the code Miguel and I have been developing for the prototype
> is here:
>
>    https://github.com/cubanismo/allocator
>
> And we've posted a port of kmscube that uses the new interfaces as a
> demonstration here:
>
>    https://github.com/cubanismo/kmscube
>
> There are still some proposed mechanisms (usage transitions mainly) that
> aren't prototyped, but I think it makes sense to start discussing
> integration while prototyping continues.

btw, I think a nice end goal would be a gralloc implementation using
this new API for sharing buffers in various use-cases.  That could
mean converting gbm-gralloc, or perhaps it means something new.

AOSP has support for mesa + upstream kernel for some devices which
also have upstream camera and/or video decoder in addition to just
GPU.. and this is where you start hitting the limits of a GBM based
gralloc.  In a lot of way, I view $new_thing as what gralloc *should*
have been, but at least it provides a way to implement a generic
gralloc.

Maybe that is getting a step ahead, there is a lot we can prototype
with kmscube.  But gralloc gets us into interesting real-world
use-cases that involve more than just GPUs.  Possibly this would be
something that linaro might be interested in getting involved with?

BR,
-R

> In addition, I'd like to note that NVIDIA is committed to providing open
> source driver implementations of these mechanisms for our hardware, in
> addition to support in our proprietary drivers.  In other words, wherever
> modifications to the nouveau kernel & userspace drivers are needed to
> implement the improved allocator mechanisms, we'll be contributing patches
> if no one beats us to it.
>
> Thanks in advance for any feedback!
>
> -James Jones
> _______________________________________________
> mesa-dev mailing list
> mesa-dev at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev