[Intel-gfx] [PATCH 7/7] drm/i915/gem: Acquire all vma/objects under reservation_ww_class

[Christian König]

Am 26.06.20 um 10:10 schrieb Chris Wilson:
> Quoting Chris Wilson (2020-06-25 18:42:41)
>> Quoting Christian König (2020-06-25 16:47:09)
>>> Am 25.06.20 um 17:10 schrieb Chris Wilson:
>>>> We have the DAG of fences, we can use that information to avoid adding
>>>> an implicit coupling between execution contexts.
>>> No, we can't. And it sounds like you still have not understood the
>>> underlying problem.
>>>
>>> See this has nothing to do with the fences itself or their DAG.
>>>
>>> When you depend on userspace to do another submission so your fence can
>>> start processing you end up depending on whatever userspace does.
>> HW dependency on userspace is explicit in the ABI and client APIs, and
>> the direct control userspace has over the HW.
>>
>>> This in turn means when userspace calls a system call (or does page
>>> fault) it is possible that this ends up in the reclaim code path.
>> We have both said the very same thing.

Then I'm really wondering why you don't come to the same conclusion :)

>>   
>>> And while we want to avoid it both Daniel and I already discussed this
>>> multiple times and we agree it is still a must have to be able to do
>>> fence waits in the reclaim code path.
>> But came to the opposite conclusion. For doing that wait harms the
>> unrelated caller and the reclaim is opportunistic. There is no need for
>> that caller to reclaim that page, when it can have any other. Why did you
>> even choose that page to reclaim? Inducing latency in the caller is a bug,
>> has been reported previously as a bug, and still considered a bug. [But at
>> the end of the day, if the system is out of memory, then you have to pick
>> a victim.]

Correct. But this is also not limited to the reclaim path as any kernel 
system call and page fault can cause a problem as well.

In other words "fence -> userspace -> page fault -> fence" or "fence -> 
userspace -> system call -> fence" can easily cause the same problem and 
that is not avoidable.

> An example
>
> Thread A				Thread B
>
> 	submit(VkCmdWaitEvents)
> 	recvfrom(ThreadB)	...	sendto(ThreadB)
> 					\- alloc_page
> 					 \- direct reclaim
> 					  \- dma_fence_wait(A)
> 	VkSetEvent()
>
> Regardless of that actual deadlock, waiting on an arbitrary fence incurs
> an unbounded latency which is unacceptable for direct reclaim.
>
> Online debugging can indefinitely suspend fence signaling, and the only
> guarantee we make of forward progress, in some cases, is process
> termination.

And exactly that is what doesn't work. You don't have any forward 
progress any more because you ran into a software deadlock.

In other words the signaling of a fence depends on the welfare of 
userspace. You can try to kill userspace, but this can wait for the 
fence you try to signal in the first place.

See the difference to a deadlock on the GPU is that you can can always 
kill a running job or process even if it is stuck with something else. 
But if the kernel is deadlocked with itself you can't kill the process 
any more, the only option left to get cleanly out of this is to reboot 
the kernel.

The only way to avoid this would be to never ever wait for the fence in 
the kernel and then your whole construct is not useful any more.

I'm running out of ideas how to explain what the problem is here....

Regards,
Christian.

> -Chris