[Mesa-dev] GBM and the Device Memory Allocator Proposals

[James Jones]

As many here know at this point, I've been working on solving issues 
related to DMA-capable memory allocation for various devices for some 
time now.  I'd like to take this opportunity to apologize for the way I 
handled the EGL stream proposals.  I understand now that the development 
process followed there was unacceptable to the community and likely 
offended many great engineers.

Moving forward, I attempted to reboot talks in a more constructive 
manner with the generic allocator library proposals & discussion forum 
at XDC 2016.  Some great design ideas came out of that, and I've since 
been prototyping some code to prove them out before bringing them back 
as official proposals.  Again, I understand some people are growing 
concerned that I've been doing this off on the side in a github project 
that has primarily NVIDIA contributors.  My goal was only to avoid 
wasting everyone's time with unproven ideas.  The intent was never to 
dump the prototype code as-is on the community and presume acceptance. 
It's just a public research project.

Now the prototyping is nearing completion, and I'd like to renew 
discussion on whether and how the new mechanisms can be integrated with 
the Linux graphics stack.

I'd be interested to know if more work is needed to demonstrate the 
usefulness of the new mechanisms, or whether people think they have 
value at this point.

After talking with people on the hallway track at XDC this year, I've 
heard several proposals for incorporating the new mechanisms:

-Include ideas from the generic allocator design into GBM.  This could 
take the form of designing a "GBM 2.0" API, or incrementally adding to 
the existing GBM API.

-Develop a library to replace GBM.  The allocator prototype code could 
be massaged into something production worthy to jump start this process.

-Develop a library that sits beside or on top of GBM, using GBM for 
low-level graphics buffer allocation, while supporting non-graphics 
kernel APIs directly.  The additional cross-device negotiation and 
sorting of capabilities would be handled in this slightly higher-level 
API before handing off to GBM and other APIs for actual allocation somehow.

-I have also heard some general comments that regardless of the 
relationship between GBM and the new allocator mechanisms, it might be 
time to move GBM out of Mesa so it can be developed as a stand-alone 
project.  I'd be interested what others think about that, as it would be 
something worth coordinating with any other new development based on or 
inside of GBM.

And of course I'm open to any other ideas for integration.  Beyond just 
where this code would live, there is much to debate about the mechanisms 
themselves and all the implementation details.  I was just hoping to 
kick things off with something high level to start.

For reference, the code Miguel and I have been developing for the 
prototype is here:

    https://github.com/cubanismo/allocator

And we've posted a port of kmscube that uses the new interfaces as a 
demonstration here:

    https://github.com/cubanismo/kmscube

There are still some proposed mechanisms (usage transitions mainly) that 
aren't prototyped, but I think it makes sense to start discussing 
integration while prototyping continues.

In addition, I'd like to note that NVIDIA is committed to providing open 
source driver implementations of these mechanisms for our hardware, in 
addition to support in our proprietary drivers.  In other words, 
wherever modifications to the nouveau kernel & userspace drivers are 
needed to implement the improved allocator mechanisms, we'll be 
contributing patches if no one beats us to it.

Thanks in advance for any feedback!

-James Jones