[PATCH 05/12] drm/amdgpu: Send no-retry XNACK for all fault types

[Felix Kuehling]

On 17-07-12 11:59 AM, Alex Deucher wrote:
> On Wed, Jul 12, 2017 at 1:40 AM, Felix Kuehling <felix.kuehling at amd.com> wrote:
>> Any comments?
>>
>> I believe this is a nice stability improvement. In case of VM faults
>> they don't take down the whole GPU with an interrupt storm. With KFD we
>> can recover without a GPU reset in many cases just by unmapping the
>> offending process' queues.
> Will this cause any problems with enabling recoverable page faults
> later?  If not,
> Acked-by: Alex Deucher <alexander.deucher at amd.com>

Like John said, this will need to be backed out when we enable
recoverable page faults. The nice thing on Vega10 is, that it's a
per-VMID setting. That will allow us for example to enable recoverable
page faults for KFD VMIDs for implementing a real HSA memory model,
without affecting the graphics VMIDs.

Still OK to add your Acked-by?

Regards,
  Felix

>
>> Regards,
>>   Felix
>>
>>
>> On 17-07-03 05:11 PM, Felix Kuehling wrote:
>>> From: Jay Cornwall <Jay.Cornwall at amd.com>
>>>
>>> A subset of VM fault types currently send retry XNACK to the client.
>>> This causes a storm of interrupts from the VM to the host.
>>>
>>> Until the storm is throttled by other means send no-retry XNACK for
>>> all fault types instead. No change in behavior to the client which
>>> will stall indefinitely with the current configuration in any case.
>>> Improves system stability under GC or MMHUB faults.
>>>
>>> Signed-off-by: Jay Cornwall <Jay.Cornwall at amd.com>
>>> Reviewed-by: Felix Kuehling <Felix.Kuehling at amd.com>
>>> ---
>>>  drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c | 3 +++
>>>  drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c  | 3 +++
>>>  2 files changed, 6 insertions(+)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
>>> index a42f483..f957b18 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
>>> @@ -206,6 +206,9 @@ static void gfxhub_v1_0_setup_vmid_config(struct amdgpu_device *adev)
>>>               tmp = REG_SET_FIELD(tmp, VM_CONTEXT1_CNTL,
>>>                               PAGE_TABLE_BLOCK_SIZE,
>>>                               adev->vm_manager.block_size - 9);
>>> +             /* Send no-retry XNACK on fault to suppress VM fault storm. */
>>> +             tmp = REG_SET_FIELD(tmp, VM_CONTEXT1_CNTL,
>>> +                                 RETRY_PERMISSION_OR_INVALID_PAGE_FAULT, 0);
>>>               WREG32_SOC15_OFFSET(GC, 0, mmVM_CONTEXT1_CNTL, i, tmp);
>>>               WREG32_SOC15_OFFSET(GC, 0, mmVM_CONTEXT1_PAGE_TABLE_START_ADDR_LO32, i*2, 0);
>>>               WREG32_SOC15_OFFSET(GC, 0, mmVM_CONTEXT1_PAGE_TABLE_START_ADDR_HI32, i*2, 0);
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c b/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c
>>> index 01918dc..b760018 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c
>>> @@ -222,6 +222,9 @@ static void mmhub_v1_0_setup_vmid_config(struct amdgpu_device *adev)
>>>               tmp = REG_SET_FIELD(tmp, VM_CONTEXT1_CNTL,
>>>                               PAGE_TABLE_BLOCK_SIZE,
>>>                               adev->vm_manager.block_size - 9);
>>> +             /* Send no-retry XNACK on fault to suppress VM fault storm. */
>>> +             tmp = REG_SET_FIELD(tmp, VM_CONTEXT1_CNTL,
>>> +                                 RETRY_PERMISSION_OR_INVALID_PAGE_FAULT, 0);
>>>               WREG32_SOC15_OFFSET(MMHUB, 0, mmVM_CONTEXT1_CNTL, i, tmp);
>>>               WREG32_SOC15_OFFSET(MMHUB, 0, mmVM_CONTEXT1_PAGE_TABLE_START_ADDR_LO32, i*2, 0);
>>>               WREG32_SOC15_OFFSET(MMHUB, 0, mmVM_CONTEXT1_PAGE_TABLE_START_ADDR_HI32, i*2, 0);
>> _______________________________________________
>> amd-gfx mailing list
>> amd-gfx at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx