Future direction of the Mesa Vulkan runtime (or "should we build a new gallium?")

[Jose Fonseca]

I don't know much about the current Vulkan driver internals to have or
provide an informed opinion on the path forward, but I'd like to share my
backwards looking perspective.

Looking back, Gallium was two things effectively:
(1) an abstraction layer, that's watertight (as in upper layers shouldn't
reach through to lower layers)
(2) an ecosystem of reusable components (draw, util, tgsi, etc.)

(1) was of course important -- and the discipline it imposed is what
enabled to great simplifications -- but it also became a straight-jacket,
as GPUs didn't stand still, and sooner or later the
see-every-hardware-as-the-same lenses stop reflecting reality.

If I had to pick one, I'd say that (2) is far more useful and practical.
Take components like gallium's draw and other util modules. A driver can
choose to use them or not.  One could fork them within Mesa source tree,
and only the drivers that opt-in into the fork would need to be
tested/adapted/etc

On the flip side, Vulkan API is already a pretty low level HW abstraction.
It's also very flexible and extensible, so it's hard to provide a
watertight abstraction underneath it without either taking the lowest
common denominator, or having lots of optional bits of functionality
governed by a myriad of caps like you alluded to.

Not sure how useful this is in practice to you, but the lesson from my POV
is that *opt-in* reusable and shared libraries are always time well spent
as they can bend and adapt with the times, whereas *no opt-out* watertight
abstractions inherently have a shelf life.

Jose

On Fri, Jan 19, 2024 at 5:30 PM Faith Ekstrand <faith at gfxstrand.net> wrote:

> Yeah, this one's gonna hit Phoronix...
>
> When we started writing Vulkan drivers back in the day, there was this
> notion that Vulkan was a low-level API that directly targets hardware.
> Vulkan drivers were these super thin things that just blasted packets
> straight into the hardware. What little code was common was small and
> pretty easy to just copy+paste around. It was a nice thought...
>
> What's happened in the intervening 8 years is that Vulkan has grown. A lot.
>
> We already have several places where we're doing significant layering.
> It started with sharing the WSI code and some Python for generating
> dispatch tables. Later we added common synchronization code and a few
> vkFoo2 wrappers. Then render passes and...
>
> https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27024
>
> That's been my project the last couple weeks: A common VkPipeline
> implementation built on top of an ESO-like interface. The big
> deviation this MR makes from prior art is that I make no attempt at
> pretending it's a layered implementation. The vtable for shader
> objects looks like ESO but takes its own path when it's useful to do
> so. For instance, shader creation always consumes NIR and a handful of
> lowering passes are run for you. It's no st_glsl_to_nir but it is a
> bit opinionated. Also, a few of the bits that are missing from ESO
> such as robustness have been added to the interface.
>
> In my mind, this marks a pretty fundamental shift in how the Vulkan
> runtime works, at least in my mind. Previously, everything was
> designed to be a toolbox where you can kind of pick and choose what
> you want to use. Also, everything at least tried to act like a layer
> where you still implemented Vulkan but you could leave out bits like
> render passes if you implemented the new thing and were okay with the
> layer. With the ESO code, you implement something that isn't Vulkan
> entrypoints and the actual entrypoints live in the runtime. This lets
> us expand and adjust the interface as needed for our purposes as well
> as sanitize certain things even in the modern API.
>
> The result is that NVK is starting to feel like a gallium driver. 🙃
>
> So here's the question: do we like this? Do we want to push in this
> direction? Should we start making more things work more this way? I'm
> not looking for MRs just yet nor do I have more reworks directly
> planned. I'm more looking for thoughts and opinions as to how the
> various Vulkan driver teams feel about this. We'll leave the detailed
> planning for the Mesa issue tracker.
>
> It's worth noting that, even though I said we've tried to keep things
> layerish, there are other parts of the runtime that look like this.
> The synchronization code is a good example. The vk_sync interface is
> pretty significantly different from the Vulkan objects it's used to
> implement. That's worked out pretty well, IMO. With as complicated as
> something like pipelines or synchronization are, trying to keep the
> illusion of a layer just isn't practical.
>
> So, do we like this? Should we be pushing more towards drivers being a
> backed of the runtime instead of a user of it?
>
> Now, before anyone asks, no, I don't really want to build a multi-API
> abstraction with a Vulkan state tracker. If we were doing this 5 years
> ago and Zink didn't already exist, one might be able to make an
> argument for pushing in that direction. However, that would add a huge
> amount of weight to the project and make it even harder to develop the
> runtime than it already is and for little benefit at this point.
>
> Here's a few other constraints on what I'm thinking:
>
> 1. I want it to still be possible for drivers to implement an
> extension without piles of runtime plumbing or even bypass the runtime
> on occasion as needed.
>
> 2. I don't want to recreate the gallium cap disaster drivers should
> know exactly what they're advertising. We may want to have some
> internal features or properties that are used by the runtime to make
> decisions but they'll be in addition to the features and properties in
> Vulkan.
>
> 3. We've got some meta stuff already but we probably want more.
> However, I don't want to force meta on folks who don't want it.
>
> The big thing here is that if we do this, I'm going to need help. I'm
> happy to do a lot of the architectural work but drivers are going to
> have to keep up with the changes and I can't take on the burden of
> moving 8 different drivers forward. I can answer questions and maybe
> help out a bit but the refactoring is going to be too much for one
> person, even if that person is me.
>
> Thoughts?
>
> ~Faith
>

-- 
This electronic communication and the information and any files transmitted 
with it, or attached to it, are confidential and are intended solely for 
the use of the individual or entity to whom it is addressed and may contain 
information that is confidential, legally privileged, protected by privacy 
laws, or otherwise restricted from disclosure to anyone else. If you are 
not the intended recipient or the person responsible for delivering the 
e-mail to the intended recipient, you are hereby notified that any use, 
copying, distributing, dissemination, forwarding, printing, or copying of 
this e-mail is strictly prohibited. If you received this e-mail in error, 
please return the e-mail to the sender, delete it from your computer, and 
destroy any printed copy of it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/mesa-dev/attachments/20240124/9148e737/attachment.htm>