[Mesa-dev] GBM and the Device Memory Allocator Proposals

Sat Nov 25 17:46:31 UTC 2017

On November 24, 2017 09:29:43 Rob Clark <robdclark at gmail.com> wrote:
>
> On Mon, Nov 20, 2017 at 8:11 PM, James Jones <jajones at nvidia.com> wrote:
>> As many here know at this point, I've been working on solving issues related
>> to DMA-capable memory allocation for various devices for some time now.  I'd
>> like to take this opportunity to apologize for the way I handled the EGL
>> stream proposals.  I understand now that the development process followed
>> there was unacceptable to the community and likely offended many great
>> engineers.
>>
>> Moving forward, I attempted to reboot talks in a more constructive manner
>> with the generic allocator library proposals & discussion forum at XDC 2016.
>> Some great design ideas came out of that, and I've since been prototyping
>> some code to prove them out before bringing them back as official proposals.
>> Again, I understand some people are growing concerned that I've been doing
>> this off on the side in a github project that has primarily NVIDIA
>> contributors.  My goal was only to avoid wasting everyone's time with
>> unproven ideas.  The intent was never to dump the prototype code as-is on
>> the community and presume acceptance. It's just a public research project.
>>
>> Now the prototyping is nearing completion, and I'd like to renew discussion
>> on whether and how the new mechanisms can be integrated with the Linux
>> graphics stack.
>>
>> I'd be interested to know if more work is needed to demonstrate the
>> usefulness of the new mechanisms, or whether people think they have value at
>> this point.
>>
>> After talking with people on the hallway track at XDC this year, I've heard
>> several proposals for incorporating the new mechanisms:
>>
>> -Include ideas from the generic allocator design into GBM.  This could take
>> the form of designing a "GBM 2.0" API, or incrementally adding to the
>> existing GBM API.
>>
>> -Develop a library to replace GBM.  The allocator prototype code could be
>> massaged into something production worthy to jump start this process.
>>
>> -Develop a library that sits beside or on top of GBM, using GBM for
>> low-level graphics buffer allocation, while supporting non-graphics kernel
>> APIs directly.  The additional cross-device negotiation and sorting of
>> capabilities would be handled in this slightly higher-level API before
>> handing off to GBM and other APIs for actual allocation somehow.
>
> tbh, I kinda see GBM and $new_thing sitting side by side.. GBM is
> still the "winsys" for running on "bare metal" (ie. kms).  And we
> don't want to saddle $new_thing with aspects of that, but rather have
> it focus on being the thing that in multiple-"device"[1] scenarious
> figures out what sort of buffer can be allocated by who for sharing.
> Ie $new_thing should really not care about winsys level things like
> cursors or surfaces.. only buffers.
>
> The mesa implementation of $new_thing could sit on top of GBM,
> although it could also just sit on top of the same internal APIs that
> GBM sits on top of.  That is an implementation detail.  It could be
> that GBM grows an API to return an instance of $new_thing for
> use-cases that involve sharing a buffer with the GPU.  Or perhaps that
> is exposed via some sort of EGL extension.  (We probably also need a
> way to get an instance from libdrm (?) for display-only KMS drivers,
> to cover cases like etnaviv sharing a buffer with a separate display
> driver.)
>
> [1] where "devices" could be multiple GPUs or multiple APIs for one or
> more GPUs, but also includes non-GPU devices like camera, video
> decoder, "image processor" (which may or may not be part of camera),
> etc, etc

I'm not quite some sure what I think about this.  I think I would like to 
see $new_thing at least replace the guts of GBM. Whether GBM becomes a 
wrapper around $new_thing or $new_thing implements the GBM API, I'm not 
sure.  What I don't think I want is to see GBM development continuing on 
it's own so we have two competing solutions.

I *think* I like the idea of having $new_thing implement GBM as a 
deprecated legacy API.  Whether that means we start by pulling GBM out into 
it's own project or we start over, I don't know.  My feeling is that the 
current dri_interface is *not* what we want which is why starting with GBM 
makes me nervous.

I need to go read through your code before I can provide a stronger or more 
nuanced opinion.  That's not going to happen before the end of the year.

>> -I have also heard some general comments that regardless of the relationship
>> between GBM and the new allocator mechanisms, it might be time to move GBM
>> out of Mesa so it can be developed as a stand-alone project.  I'd be
>> interested what others think about that, as it would be something worth
>> coordinating with any other new development based on or inside of GBM.
>
> +1
>
> We already have at least a couple different non-mesa implementations
> of GBM (which afaict tend to lag behind mesa's GBM and cause
> headaches).
>
> The extracted part probably isn't much more than a header and shim.
> But probably does need to grow some versioning for the backend to know
> if, for example, gbm->bo_map() is supported.. at least it could
> provide stubs that return an error, rather than having link-time fail
> if building something w/ $vendor's old gbm implementation.
>
>> And of course I'm open to any other ideas for integration.  Beyond just
>> where this code would live, there is much to debate about the mechanisms
>> themselves and all the implementation details.  I was just hoping to kick
>> things off with something high level to start.
>
> My $0.02, is that the place where devel happens and place to go for
> releases could be different.  Either way, I would like to see git tree
> for tagged release versions live on fd.o and use the common release
> process[2] for generating/uploading release tarballs that distros can
> use.

Agreed.  I think fd.o is the right place for such a project to live.  We 
can have mirrors on GitHub and other places but fd.o is where Linux 
graphics stack development currently happens.

> [2] https://cgit.freedesktop.org/xorg/util/modular/tree/release.sh
>
>> For reference, the code Miguel and I have been developing for the prototype
>> is here:
>>
>>    https://github.com/cubanismo/allocator
>>
>> And we've posted a port of kmscube that uses the new interfaces as a
>> demonstration here:
>>
>>    https://github.com/cubanismo/kmscube
>>
>> There are still some proposed mechanisms (usage transitions mainly) that
>> aren't prototyped, but I think it makes sense to start discussing
>> integration while prototyping continues.
>
> btw, I think a nice end goal would be a gralloc implementation using
> this new API for sharing buffers in various use-cases.  That could
> mean converting gbm-gralloc, or perhaps it means something new.
>
> AOSP has support for mesa + upstream kernel for some devices which
> also have upstream camera and/or video decoder in addition to just
> GPU.. and this is where you start hitting the limits of a GBM based
> gralloc.  In a lot of way, I view $new_thing as what gralloc *should*
> have been, but at least it provides a way to implement a generic
> gralloc.

+100

> Maybe that is getting a step ahead, there is a lot we can prototype
> with kmscube.  But gralloc gets us into interesting real-world
> use-cases that involve more than just GPUs.  Possibly this would be
> something that linaro might be interested in getting involved with?
>
> BR,
> -R
>
>> In addition, I'd like to note that NVIDIA is committed to providing open
>> source driver implementations of these mechanisms for our hardware, in
>> addition to support in our proprietary drivers.  In other words, wherever
>> modifications to the nouveau kernel & userspace drivers are needed to
>> implement the improved allocator mechanisms, we'll be contributing patches
>> if no one beats us to it.
>>
>> Thanks in advance for any feedback!
>>
>> -James Jones
>> _______________________________________________
>> mesa-dev mailing list
>> mesa-dev at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
> _______________________________________________
> mesa-dev mailing list
> mesa-dev at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev