[PATCH 00/35] Add HMM-based SVM memory manager to KFD

Thu Jan 14 12:19:16 UTC 2021

Am 14.01.21 um 06:34 schrieb Felix Kuehling:
> Am 2021-01-11 um 11:29 a.m. schrieb Daniel Vetter:
>> On Fri, Jan 08, 2021 at 12:56:24PM -0500, Felix Kuehling wrote:
>>> Am 2021-01-08 um 11:53 a.m. schrieb Daniel Vetter:
>>>> On Fri, Jan 8, 2021 at 5:36 PM Felix Kuehling <felix.kuehling at amd.com> wrote:
>>>>> Am 2021-01-08 um 11:06 a.m. schrieb Daniel Vetter:
>>>>>> On Fri, Jan 8, 2021 at 4:58 PM Felix Kuehling <felix.kuehling at amd.com> wrote:
>>>>>>> Am 2021-01-08 um 9:40 a.m. schrieb Daniel Vetter:
>>>>>>>> On Thu, Jan 07, 2021 at 11:25:41AM -0500, Felix Kuehling wrote:
>>>>>>>>> Am 2021-01-07 um 4:23 a.m. schrieb Daniel Vetter:
>>>>>>>>>> On Wed, Jan 06, 2021 at 10:00:52PM -0500, Felix Kuehling wrote:
>>>>>>>>>>> This is the first version of our HMM based shared virtual memory manager
>>>>>>>>>>> for KFD. There are still a number of known issues that we're working through
>>>>>>>>>>> (see below). This will likely lead to some pretty significant changes in
>>>>>>>>>>> MMU notifier handling and locking on the migration code paths. So don't
>>>>>>>>>>> get hung up on those details yet.
>>>>>>>>>>>
>>>>>>>>>>> But I think this is a good time to start getting feedback. We're pretty
>>>>>>>>>>> confident about the ioctl API, which is both simple and extensible for the
>>>>>>>>>>> future. (see patches 4,16) The user mode side of the API can be found here:
>>>>>>>>>>> https://github.com/RadeonOpenCompute/ROCT-Thunk-Interface/blob/fxkamd/hmm-wip/src/svm.c
>>>>>>>>>>>
>>>>>>>>>>> I'd also like another pair of eyes on how we're interfacing with the GPU VM
>>>>>>>>>>> code in amdgpu_vm.c (see patches 12,13), retry page fault handling (24,25),
>>>>>>>>>>> and some retry IRQ handling changes (32).
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Known issues:
>>>>>>>>>>> * won't work with IOMMU enabled, we need to dma_map all pages properly
>>>>>>>>>>> * still working on some race conditions and random bugs
>>>>>>>>>>> * performance is not great yet
>>>>>>>>>> Still catching up, but I think there's another one for your list:
>>>>>>>>>>
>>>>>>>>>>   * hmm gpu context preempt vs page fault handling. I've had a short
>>>>>>>>>>     discussion about this one with Christian before the holidays, and also
>>>>>>>>>>     some private chats with Jerome. It's nasty since no easy fix, much less
>>>>>>>>>>     a good idea what's the best approach here.
>>>>>>>>> Do you have a pointer to that discussion or any more details?
>>>>>>>> Essentially if you're handling an hmm page fault from the gpu, you can
>>>>>>>> deadlock by calling dma_fence_wait on a (chain of, possibly) other command
>>>>>>>> submissions or compute contexts with dma_fence_wait. Which deadlocks if
>>>>>>>> you can't preempt while you have that page fault pending. Two solutions:
>>>>>>>>
>>>>>>>> - your hw can (at least for compute ctx) preempt even when a page fault is
>>>>>>>>    pending
>>>>>>> Our GFXv9 GPUs can do this. GFXv10 cannot.
>>>>>> Uh, why did your hw guys drop this :-/
>>> Performance. It's the same reason why the XNACK mode selection API
>>> exists (patch 16). When we enable recoverable page fault handling in the
>>> compute units on GFXv9, it costs some performance even when no page
>>> faults are happening. On GFXv10 that retry fault handling moved out of
>>> the compute units, so they don't take the performance hit. But that
>>> sacrificed the ability to preempt during page faults. We'll need to work
>>> with our hardware teams to restore that capability in a future generation.
>> Ah yes, you need to stall in more points in the compute cores to make sure
>> you can recover if the page fault gets interrupted.
>>
>> Maybe my knowledge is outdated, but my understanding is that nvidia can
>> also preempt (but only for compute jobs, since oh dear the pain this would
>> be for all the fixed function stuff). Since gfx10 moved page fault
>> handling further away from compute cores, do you know whether this now
>> means you can do page faults for (some?) fixed function stuff too? Or
>> still only for compute?
> I'm not sure.
>
>
>> Supporting page fault for 3d would be real pain with the corner we're
>> stuck in right now, but better we know about this early than later :-/
> I know Christian hates the idea.

Well I don't hate the idea. I just don't think that this will ever work 
correctly and performant.

A big part of the additional fun is that we currently have a mix of HMM 
capable engines (3D, compute, DMA) and not HMM capable engines (display, 
multimedia etc..).

> We know that page faults on GPUs can be
> a huge performance drain because you're stalling potentially so many
> threads and the CPU can become a bottle neck dealing with all the page
> faults from many GPU threads. On the compute side, applications will be
> optimized to avoid them as much as possible, e.g. by pre-faulting or
> pre-fetching data before it's needed.
>
> But I think you need page faults to make overcommitted memory with user
> mode command submission not suck.

Yeah, completely agree.

The only short term alternative I see is to have an IOCTL telling the 
kernel which memory is currently in use. And that is complete nonsense 
cause it kills the advantage why we want user mode command submission in 
the first place.

Regards,
Christian.

>>>>>> I do think it can be rescued with what I call gang scheduling of
>>>>>> engines: I.e. when a given engine is running a context (or a group of
>>>>>> engines, depending how your hw works) that can cause a page fault, you
>>>>>> must flush out all workloads running on the same engine which could
>>>>>> block a dma_fence (preempt them, or for non-compute stuff, force their
>>>>>> completion). And the other way round, i.e. before you can run a legacy
>>>>>> gl workload with a dma_fence on these engines you need to preempt all
>>>>>> ctxs that could cause page faults and take them at least out of the hw
>>>>>> scheduler queue.
>>>>> Yuck! But yeah, that would work. A less invasive alternative would be to
>>>>> reserve some compute units for graphics contexts so we can guarantee
>>>>> forward progress for graphics contexts even when all CUs working on
>>>>> compute stuff are stuck on page faults.
>>>> Won't this hurt compute workloads? I think we need something were at
>>>> least pure compute or pure gl/vk workloads run at full performance.
>>>> And without preempt we can't take anything back when we need it, so
>>>> would have to always upfront reserve some cores just in case.
>>> Yes, it would hurt proportionally to how many CUs get reserved. On big
>>> GPUs with many CUs the impact could be quite small.
>> Also, we could do the reservation only for the time when there's actually
>> a legacy context with normal dma_fence in the scheduler queue. Assuming
>> that reserving/unreserving of CUs isn't too expensive operation. If it's
>> as expensive as a full stall probably not worth the complexity here and
>> just go with a full stall and only run one or the other at a time.
>>
>> Wrt desktops I'm also somewhat worried that we might end up killing
>> desktop workloads if there's not enough CUs reserved for these and they
>> end up taking too long and anger either tdr or worse the user because the
>> desktop is unuseable when you start a compute job and get a big pile of
>> faults. Probably needs some testing to see how bad it is.
>>
>>> That said, I'm not sure it'll work on our hardware. Our CUs can execute
>>> multiple wavefronts from different contexts and switch between them with
>>> fine granularity. I'd need to check with our HW engineers whether this
>>> CU-internal context switching is still possible during page faults on
>>> GFXv10.
>> You'd need to do the reservation for all contexts/engines which can cause
>> page faults, otherewise it'd leak.
> All engines that can page fault and cannot be preempted during faults.
>
> Regards,
>    Felix
>