[Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal

Christian König ckoenig.leichtzumerken at gmail.com
Tue Apr 20 18:00:27 UTC 2021

Am 20.04.21 um 19:44 schrieb Daniel Stone:
> Hi,
> On Tue, 20 Apr 2021 at 16:46, Jason Ekstrand <jason at jlekstrand.net 
> <mailto:jason at jlekstrand.net>> wrote:
>     It's still early in the morning here and I'm not awake yet so sorry if
>     this comes out in bits and pieces...
> No problem, it's helpful. If I weren't on this thread I'd be 
> attempting to put together a 73-piece chest of drawers whose 
> instructions are about as clear as this so far, so I'm in the right 
> head space anyway.
>     IMO, there are two problems being solved here which are related in
>     very subtle and tricky ways.  They're also, admittedly, driver
>     problems, not really winsys problems.  Unfortunately, they may have
>     winsys implications.
> Yeah ... bingo.
>     First, is better/real timelines for Vulkan and compute. [...]
>     We also want something like this for compute workloads. [...]
> Totally understand and agree with all of this. Memory fences seem like 
> a good and useful primitive here.

Completely agree.

>     The second biting issue is that, in the current kernel implementation
>     of dma-fence and dma_resv, we've lumped internal synchronization for
>     memory management together with execution synchronization for
>     userspace dependency tracking.  And we have no way to tell the
>     difference between the two internally.  Even if user space is passing
>     around sync_files and trying to do explicit sync, once you get inside
>     the kernel, they're all dma-fences and it can't tell the difference.
> Funny, because 'lumped [the two] together' is exactly the crux of my 
> issues ...
>     If we move
> Stop here, because ...
>     to a more userspace-controlled synchronization model with
>     wait-before-signal and no timeouts unless requested, regardless of the
>     implementation, it plays really badly dma-fence.  And, by "badly" I
>     mean the two are nearly incompatible.
> I would go further than that, and say completely, fundamentally, 
> conceptually, incompatible.


>     From a user space PoV, it means
>     it's tricky to provide the finite time dma-fence guarantee. From a
>     kernel PoV, it's way worse.  Currently, the way dma-fence is
>     constructed, it's impossible to deadlock assuming everyone follows the
>     rules.  The moment we allow user space to deadlock itself and allow
>     those deadlocks to leak into the kernel, we have a problem. Even if
>     we throw in some timeouts, we still have a scenario where user space
>     has one linearizable dependency graph for execution synchronization
>     and the kernel has a different linearizable dependency graph for
>     memory management and, when you smash them together, you may have
>     cycles in your graph.
>     So how do we sort this all out?  Good question.  It's a hard problem.
>     Probably the hardest problem here is the second one: the intermixing
>     of synchronization types.  Solving that one is likely going to require
>     some user space re-plumbing because all the user space APIs we have
>     for explicit sync are built on dma-fence.
> Gotcha.
> Firstly, let's stop, as you say, lumping things together. Timeline 
> semaphores and compute's GPU-side spinlocks etc, are one thing. I 
> accept those now have a hard requirement on something like memory 
> fences, where any responsibility is totally abrogated. So let's run 
> with that in our strawman: Vulkan compute & graphics & transfer queues 
> all degenerate to something spinning (hopefully GPU-assisted gentle 
> spin) on a uint64 somewhere. The kernel has (in the general case) no 
> visibility or responsibility into these things. Fine - that's one side 
> of the story.

Exactly, yes.

> But winsys is something _completely_ different. Yes, you're using the 
> GPU to do things with buffers A, B, and C to produce buffer Z. Yes, 
> you're using vkQueuePresentKHR to schedule that work. Yes, Mutter's 
> composition job might depend on a Chromium composition job which 
> depends on GTA's render job which depends on GTA's compute job which 
> might take a year to complete. Mutter's composition job needs to 
> complete in 'reasonable' (again, FSVO) time, no matter what. The two 
> are compatible.
> How? Don't lump them together. Isolate them aggressively, and 
> _predictably_ in a way that you can reason about.
> What clients do in their own process space is their own 
> business. Games can deadlock themselves if they get wait-before-signal 
> wrong. Compute jobs can run for a year. Their problem. Winsys is not 
> that, because you're crossing every isolation boundary possible. 
> Process, user, container, VM - every kind of privilege boundary. Thus 
> far, dma_fence has protected us from the most egregious abuses by 
> guaranteeing bounded-time completion; it also acts as a sequencing 
> primitive, but from the perspective of a winsys person that's of 
> secondary importance, which is probably one of the bigger disconnects 
> between winsys people and GPU driver people.

Finally somebody who understands me :)

Well the question is then how do we get winsys and your own process 
space together then?

> Anyway, one of the great things about winsys (there are some! trust 
> me) is we don't need to be as hopelessly general as for game engines, 
> nor as hyperoptimised. We place strict demands on our clients, and we 
> literally kill them every single time they get something wrong in a 
> way that's visible to us. Our demands on the GPU are so embarrassingly 
> simple that you can run every modern desktop environment on GPUs which 
> don't have unified shaders. And on certain platforms who don't share 
> tiling formats between texture/render-target/scanout ... and it all 
> still runs fast enough that people don't complain.

Ignoring everything below since that is the display pipeline I'm not 
really interested in. My concern is how to get the buffer from the 
client to the server without allowing the client to get the server into 

My thinking is still to use timeouts to acquire texture locks. E.g. when 
the compositor needs to access texture it grabs a lock and if that lock 
isn't available in less than 20ms whoever is holding it is killed hard 
and the lock given to the compositor.

It's perfectly fine if a process has a hung queue, but if it tries to 
send buffers which should be filled by that queue to the compositor it 
just gets a corrupted window content.


> We're happy to bear the pain of being the ones setting strict and 
> unreasonable expectations. To me, this 'present ioctl' falls into the 
> uncanny valley of the kernel trying to bear too much of the weight to 
> be tractable, whilst not bearing enough of the weight to be useful for 
> winsys.
> So here's my principles for a counter-strawman:
> Remove the 'return fence'. Burn it with fire, do not look back. Modern 
> presentation pipelines are not necessarily 1:1, they are not 
> necessarily FIFO (as opposed to mailbox), and they are not necessarily 
> round-robin either. The current proposal provides no tangible benefits 
> to modern userspace, and fixing that requires either hobbling 
> userspace to remove capability and flexibility (ironic given that the 
> motivation for this is all about userspace flexibility?), or pushing 
> so much complexity into the kernel that we break it forever (you can't 
> compile Mutter's per-frame decision tree into eBPF).
> Give us a primitive representing work completion, so we can keep 
> optimistically pipelining operations. We're happy to pass around 
> explicit-synchronisation tokens (dma_fence, drm_syncobj, drm_newthing, 
> whatever it is): plumbing through a sync token to synchronise 
> compositor operations against client operations in both directions is 
> just a matter of boring typing.
> Make that primitive something that is every bit as usable across 
> subsystems as it is across processes. It should be a lowest common 
> denominator for middleware that ultimately provokes GPU execbuf, KMS 
> commit, and media codec ops; currently that would be both wait and 
> signal for all of VkSemaphore, EGLSyncKHR, KMS fence, V4L (D)QBUF, and 
> VA-API {en,de}code ops. It must be exportable to and importable from 
> an FD, which can be poll()ed on and read(). GPU-side visibility for 
> late binding is nice, but not at all essential.
> Make that primitive complete in 'reasonable' time, no matter what. 
> There will always be failures in extremis, no matter what the design: 
> absent hard-realtime principles from hardware all the way up to 
> userspace, something will always be able to fail somewhere: 
> non-terminating GPU work, actual GPU hang/reset, GPU queue DoSed, CPU 
> scheduler, I/O DoSed. As long as the general case is bounded-time 
> completion, each of these can be mitigated separately as long as 
> userspace has enough visibility into the underlying mechanics, and 
> cares enough to take meaningful action on it.
> And something more concrete:
> dma_fence.
> This already has all of the properties described above. Kernel-wise, 
> it already devolves to CPU-side signaling when it crosses device 
> boundaries. We need to support it roughly forever since it's been 
> plumbed so far and so wide. Any primitive which is acceptable for 
> winsys-like usage which crosses so many 
> device/subsystem/process/security boundaries has to meet the same 
> requirements. So why reinvent something which looks so similar, and 
> has the same requirements of the kernel babysitting completion, 
> providing little to no benefit for that difference?
> It's not usable for complex usecases, as we've established, but winsys 
> is not that usecase. We can draw a hard boundary between the two 
> worlds. For example, a client could submit an infinitely deep CS -> 
> VS/FS/etc job chain with potentially-infinite completion, with the FS 
> output being passed to the winsys for composition. Draw the line 
> post-FS: export a dma_fence against FS completion. But instead of this 
> being based on monitoring the _fence_ per se, base it on monitoring 
> the job; if the final job doesn't retire in reasonable time, signal 
> the fence and signal (like, SIGKILL, or just tear down the context and 
> permanently -EIO, whatever) the client. Maybe for future hardware that 
> would be the same thing - the kernel setting a timeout and comparing a 
> read on a particular address against a particular value - but the 
> 'present fence' proposal seems like it requires exactly this anyway.
> That to me is the best compromise. We allow clients complete arbitrary 
> flexibility, but as soon as they vkQueuePresentKHR, they're crossing a 
> boundary out of happy fun GPU land and into strange hostile winsys 
> land. We've got a lot of practice at being the bad guys who hate users 
> and are always trying to ruin their dreams, so we'll happily wear the 
> impact of continuing to do that. In doing so, we collectively don't 
> have to invent a third new synchronisation primitive (to add to 
> dma_fence and drm_syncobj) and a third new synchronisation model 
> (implicit sync, explicit-but-bounded sync, 
> explicit-and-maybe-unbounded sync) to support this, and we don't have 
> to do an NT4 where GDI was shoved into the kernel.
> It doesn't help with the goal of ridding dma_fence from the kernel, 
> but it does very clearly segregate the two worlds. Drawing that hard 
> boundary would allow drivers to hyperoptimise for clients which want 
> to be extremely clever and agile and quick because they're sailing so 
> close to the wind that they cannot bear the overhead of dma_fence, 
> whilst also providing the guarantees we need when crossing isolation 
> boundaries. In the latter case, the overhead of bouncing into a 
> less-optimised primitive is totally acceptable because it's not even 
> measurable: vkQueuePresentKHR requires client CPU activity -> kernel 
> IPC -> compositor CPU activity -> wait for repaint cycle -> prepare 
> scene -> composition, against which dma_fence overhead isn't and will 
> never be measurable (even if it doesn't cross device/subsystem 
> boundaries, which it probably does). And the converse for 
> vkAcquireNextImageKHR.
> tl;dr: we don't need to move winsys into the kernel, winsys and 
> compute don't need to share sync primitives, the client/winsys 
> boundary does need to have a sync primitive does need strong and 
> onerous guarantees, and that transition can be several orders of 
> magnitude less efficient than intra-client sync primitives
> Shoot me down. :)
> Cheers,
> Daniel
> _______________________________________________
> mesa-dev mailing list
> mesa-dev at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/mesa-dev/attachments/20210420/d1ebdeb3/attachment-0001.htm>

More information about the mesa-dev mailing list