completely rework the dma_resv semantic

[Daniel Vetter]

On Tue, Nov 23, 2021 at 03:20:45PM +0100, Christian König wrote:
> Hi guys,
> 
> as discussed before this set of patches completely rework the dma_resv semantic
> and spreads the new handling over all the existing drivers and users.
> 
> First of all this drops the DAG approach because it requires that every single
> driver implements those relatively complicated rules correctly and any
> violation of that immediately leads to either corruption of freed memory or
> even more severe security problems.
> 
> Instead we just keep all fences around all the time until they are signaled.
> Only fences with the same context are assumed to be signaled in the correct
> order since this is exercised elsewhere as well. Replacing fences is now only
> supported for hardware mechanism like VM page table updates where the hardware
> can guarantee that the resource can't be accessed any more.
> 
> Then the concept of a single exclusive fence and multiple shared fences is
> dropped as well.
> 
> Instead the dma_resv object is now just a container for dma_fence objects where
> each fence has associated usage flags. Those use flags describe how the
> operation represented by the dma_fence object is using the resource protected
> by the dma_resv object. This allows us to add multiple fences for each usage
> type.
> 
> Additionally to the existing WRITE/READ usages this patch set also adds the new
> KERNEL and OTHER usages. The KERNEL usages is used in cases where the kernel
> needs to do some operation with the resource protected by the dma_resv object,
> like copies or clears. Those are mandatory to wait for when dynamic memory
> management is used.
> 
> The OTHER usage is for cases where we don't want that the operation represented
> by the dma_fence object participate in any implicit sync but needs to be
> respected by the kernel memory management. Examples for those are VM page table
> updates and preemption fences.
> 
> While doing this the new implementation cleans up existing workarounds all over
> the place, but especially amdgpu and TTM. Surprisingly I also found two use
> cases for the KERNEL/OTHER usage in i915 and Nouveau, those might need more
> thoughts.
> 
> In general the existing functionality should been preserved, the only downside
> is that we now always need to reserve a slot before adding a fence. The newly
> added call to the reservation function can probably use some more cleanup.
> 
> TODOs: Testing, testing, testing, doublechecking the newly added
> kerneldoc for any typos.
> 
> Please review and/or comment,

I like.

Unfortunately also massively burried, but I really like. I think the past
few months (years?) of discussions and bikeshed have been worth it, this
looks tidy and clear in semantics and in how drivers use it all.

Ofc this will take some time to review/test in detail and land, but I
think next steps would be to resurrect Jason's explicit dma-buf fence
import/export series (should also clean up nicely I think), and then roll
out the new fence semantics to a few vk/compute stacks? I think especially
for vk what we want is that normal CS only ever uses OTHER, and any
implicit sync that needs to happen for winsys buffers is done through the
import/export ioctls. GL might need something slightly different, but
normally there's not many shared buffers, so doing a pile of ioctl calls
for implicit synced buffers seems fine. But perhaps GL does want a new CS
ioctl flag.

Cheers, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch